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PREFACE 

Many  studies  are  not  as  effective  as  they  could  be  because  the  analyst  does  not 
thoroughly  understand  the  needs  of  those  using  the  analyses  to  make  decisions.  Likewise,  other 
studies  have  only  a  marginal  impact  because  the  supported  senior  official  does  not  have  a  clear 
understanding  of  the  limits  and  capabilities  of  analysis.  In  an  attempt  to  help  increase  the 
usefulness  of  studies,  IDA  sponsored  a  Central  Research  Project  to  develop  an  instructional 
course  based  on  case  studies  of  military  analysis  that  could  be  incorporated  into  DoD's  senior 
executive  training  schools  and  analyst  educational  curricula. 

This  case  study  is  the  second  of  a  series1  of  papers  that  describe  already  completed  studies  that 
may  prove  useful  to  those  who  use  analyses  to  support  major  decisions,  those  who  design  and 
conduct  analyses,  or  those  who  review  analytical  studies. 

The  general  approach  taken  in  all  these  papers  is  to  (1)  explain  the  issues  underlying  an 
important  decision  in  the  recent  past,  (2)  describe  how  the  issues  were  studied  analytically,  and 
(3)  indicate  the  decisions  that  ensued.  But  these  papers  provide  more  than  just  a  summary  of  the 
findings  reported  in  earlier  work.  They  also  explore  the  different  perspectives  of  the  various 
players,  how  perceptions  changed  over  time,  unexpected  turns  in  the  analyses,  disappointments, 
blind  alleys,  lessons  learned,  and  other  features  that  are  a  part  of  any  real-life  study,  although  they 
are  rarely  documented  in  final  reports.  Familiarity  with  these  organic,  dynamically  changing 
features  can  help  decisionmakers  and  analysts  alike  better  realize  the  value  and  limits  of  analyses. 
At  the  same  time,  senior  managers,  equipped  with  realistic  expectations  and  an  awareness  of 
analytic  constraints  on  analyses,  can  better  avoid  predictable  pitfalls  and  help  structure  the 
analysis  so  that  it  generates  the  insights  needed  to  inform  their  decisions. 

This  case  study  is  for  the  Deep  Attack  Weapons  Mix  Study  conducted  from  1995  to 
1997.2  It  provides  basic  background  to  describe  the  key  issues  and  data  used  in  the  conduct  of  the 
analysis.  The  case  study  does  not  cover  every  aspect  of  the  analysis,  which  would  require  a  much 
more  lengthy  treatise.  Also,  the  description  below  simplifies  some  aspects  of  the  analysis  to 
avoid  the  use  of  classified  inputs  and  results.  The  focus  of  this  paper  is  on  the  overall  study 
process. 

The  case  study  ends  with  a  series  of  questions  that  could  be  used  as  the  basis  for 
discussion  in  the  classroom. 

Finally,  we  would  like  to  thank  the  reviewers,  Dr.  Ron  Enlow  and  Dr.  Geoffrey  Koretsky, 
for  their  useful  insight  and  helpful  suggestions. 


1  The  first  case  study  developed  was  for  the  C-17  Cost  and  Operational  Effectiveness  Analysis  (COEA). 
See  IDA  Document  D-2688,  C-17  COEA  Case  Study. 

2  There  was  a  follow-on  to  DAWMS  called  The  Deep  Attack  Study  (DAS)  that  updated  the  databases  and 
improved  the  models. 
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I.  BACKGROUND 


A.  INTRODUCTION 

In  1995  the  Report  of  the  Commission  on  Roles  and  Missions  (CORM)  of  the 
Armed  Forces1  recommended  that  the  Department  of  Defense  (DoD)  conduct  a  study 
...focused  on  finding  the  appropriate  combination  and  quantities  of  deep  attack " 
capabilities  curren  tly  fielded  and  under  developmen  t  by  all  Services. 

The  CORM  report  stated  the  following  position  on  the  Deep  Attack  issue: 

...CINCs  have  available  several  different  weapon  systems  that  can  attack 
land  and  sea  targets  at  varying  ranges.  The  Services  field  a  mix  of  land- 
based  ballistic  missiles,  sea-based  cruise  missiles,  and  a  growing 
inventory  of precision  guided  weapons  and  standoff  weapons  delivered  by 
aircraft.  All  of  these  capabilities  are  useful.  No  CINC  proposed 
eliminating  any  of  these  capabilities....”  However,  it  is  not  clear  that 
DOD  has  the  correct  balance  of  these  various  weapons.  Currently,  no  one 
in  DOD  has  specific  responsibility  for  specifying  the  overall  number  and 
mix  of  deep  attack  systems...  Consequently,  we  [the  CORM]  recommend 
prompt  initiation  of  a  DOD-wide  cost-effectiveness  study  focused  on 
finding  the  appropriate  combination  and  quantities  of  deep  attack 
capabilities  currently  fielded  and  under  development  by  all  Services.  Only 
by  approaching  capabilities  in  the  aggregate,  from  the  CINCs' perspective 
rather  than  the  Sendees',  can  this  particular  'who  needs  what'  question  be 
answered. 

The  DoD  Authorization  Bill  for  Fiscal  Year  1996  required  the  Secretary  of 
Defense  to  perform  an  analysis  of  the  full  range  of  precision-guided  munitions  (PGMs)  in 
production  and  in  research,  development,  test,  and  evaluation  (RDT&E)  in  order  to 


1  Directions  for  Defense,  Commission  on  Roles  and  Missions  (CORM),  May  1995,  UNCLASSIFIED. 

2  Deep  Attack  refers  to  all  of  the  combat  activity  beyond  the  close  battle  area.  A  formal  definition  is 
provided  in  section  D. 
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determine  the  need  for  accelerated  funding  of  PGMs  and,  if  needed,  determine  priorities 
and  procurement  objectives. 

The  above  guidance  resulted  in  the  Deep  Attack  Weapons  Mix  Study 
(DAWMS).  It  was  the  major  study  conducted  by  the  Department  of  Defense  during  the 
2-year  period  starting  in  August  1995.  Its  objective  was  to  ...determine  the  most  cost- 
effective  mix  (quantity  and  composition)  of  air-to-surface  and  surface-to-surface 
munitions  for  deep  attack  operations.  Each  year  the  Services  individually  develop 
recommended  weapons  purchases  based  on  targets  allocated  to  them  in  the  various 
approved  CINC  warplans.  DAWMS  was  the  first  coordinated  attempt  by  the  Department 
to  jointly  influence  the  planned  weapon  mix  across  the  Services.  It  involved  all  four 
Services  and  was  one  of  the  few  studies  to  be  overseen  by  the  Joint  Requirements 
Oversight  Council  (JROC).  Because  it  departed  from  the  traditional  approach  to 
allocating  and  purchasing  weapons,  and  because  it  addressed  equities  of  all  CINCs, 
Services,  and  those  portions  of  the  Office  of  the  Secretary  of  Defense  (OSD)  with  plans 
and  acquisition  oversight  responsibilities,  the  study  became  highly  controversial. 

B.  THE  PARTICIPANTS 

1.  Overview 

How  the  study  team  is  organized  and  the  interface  the  study  leaders  have  with  the 
customers  (senior  DoD  officials)  can  play  a  major  part  in  the  success  and  impact  of  the 
analysis.  The  study  organizational  structure  is  summarized  in  Figure  1  and  then 
discussed  in  greater  detail  in  this  section. 

2.  Study  Team 

The  study  leader  was  an  0-6  from  the  Joint  Staff  (J-8)  whose  deputy  was  an  0-6 
from  OSD  policy.  Over  the  course  of  the  study  both  the  study  leader  and  the  deputy 
transitioned  due  to  the  normal  rotation  associated  with  military  change  of  assignments. 
Most  of  the  direct  analytical  support  was  provided  by  analysts  from  the  Institute  for 


This  case  study  concerns  only  DAWMS,  Part  1 .  There  was  a  DAWMS,  Part  2.  The  objective  of  Part 
2  was  to  determine  the  appropriate  force  size  and  mix  for  deep  attack  operations.  More  specifically,  it 
examined  trade-offs  among  long-range  bombers  (including  additional  B-2s),  land-  and  sea-based 
tactical  aircraft,  and  missiles  that  are  used  in  deep  attack  operations. 
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Defense  Analyses  (IDA)  and  the  Joint  Staff  (J-8),  although  each  of  the  Services 
conducted  parallel  study  activities  and  made  much  of  their  work  available  to  the  IDA 
study  team  members. 


Figure  1.  Deep  Attack/Weapons  Mix  Study  Organization 

The  study  involved  the  use  of  a  methodology  intended  to  extend  beyond  single 
Service  optimization  and  depended  on  the  development  of  a  detailed  database  to  which 
all  the  Services  contributed.  Several  groups4  helped  facilitate  this  data  collection  and 
ensure  consistency  across  Service  inputs: 

•  The  Target  Value  Group  coordinated  with  Service  planners  to  establish  values 
for  killing  red  targets  as  a  function  of  target  type,  location,  and  time  (phase  of 
the  war). 


4 


IDA  analysts  played  a  major  role  in  all  parts  of  the  study  and  had  key  members  attending  the  Working 
Group,  Steering  Group. 
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•  The  Weapon  Effectiveness  Group  developed  consistent  definitions  of  the 
targets  together  with  estimates  of  the  effects  each  weapon  type  would  have 
against  each  target  type. 

•  The  Command,  Control,  Communication,  Computers,  Intelligence, 
Surveillance,  and  Reconnaissance  (C4ISR)  Group  provided  both  the  approach 
and  the  data  needed  to  incorporate  C4ISR  effects  in  the  study.  In  fact, 
capturing  these  C4ISR  effects  was  a  major  reason  for  the  study  since 
individual  Service  weapon  mix  studies  did  not  include  an  in-depth  treatment 
of  how  C4ISR  might  impact  weapons  and  their  effectiveness  in  various 
applications.  It  was  one  of  the  first  to  fully  integrate  C4ISR  factors  into  a 
campaign  analysis. 

•  The  Cost  Group  developed  the  cost  estimates  for  all  of  the  weapons. 

Another  group,  called  the  Production  Analysis  Team,  was  established  to  oversee 
the  analysis  portion  of  the  study.  It  was  co-chaired  by  an  0-6  from  J-8  and  an  analyst 
from  IDA,  and  it  met  at  least  weekly  during  most  of  the  study.  Members  included 
representatives  from  all  four  Services  who  were  able  to  review  firsthand  the  methodology 
used,  the  exact  data  being  entered  into  the  models,  the  results  produced,  and  analytic 
insights.  The  Service  members  approved  the  use  of  all  data  from  their  Service  and  often 
suggested  changes  to  the  analytical  approaches  or  sensitivities  to  be  explored  to  help 
reviewers  gain  confidence  in  the  results. 

Another  group  was  established  to  ensure  that  the  study  was  consistent  with 
Service  doctrine  and  operational  practices.  This  group  was  called  the  Battle  Staff  and 
consisted  of  senior  0-6s  from  each  of  the  Services.  In  addition  to  helping  translate 
Service  doctrine  and  tactical  concepts  into  model  inputs  for  the  various  simulations  used 
in  the  study,  these  members  helped  establish  such  important  inputs  as  the  pace  of  the  joint 
suppression  of  enemy  air  defense  (JSEAD)  campaign. 

3.  Oversight 

The  study  structure  included  three  levels  of  oversight:  working  group,  steering 
group  and  senior  advisory  group  (SAG).  The  study  director  reported  to  the  working  and 
steering  groups  every  4-6  weeks  during  the  middle  and  later  phases  of  the  study.  The 
Working  Group  was  co-chaired  by  the  study  leader  from  J-8  and  representatives  from  the 
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policy,  acquisition,  and  program  analysis  and  evaluation  offices.  Members  were 
primarily  senior  0-6s  or  their  equivalents  from  OSD,  the  Joint  Staff,  the  Services,  DIA, 
and  the  CINC  staffs. 

The  Steering  Group  had  similar  membership  except  that  the  members  were 
typically  0-8s  or  their  civilian  equivalents,  chaired  by  the  Assistant  Secretary  of  Defense 
for  Strategy  and  Requirements. 

The  Senior  Advisory  Group  met  every  3-4  months  over  the  course  of  the  study 
and  more  frequently  toward  the  end.  It  was  co-chaired  by  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology  and  the  Vice  Chairman  of  the  Joint  Chiefs  of 
Staff.  Members  included  the  Service  Vice  Chiefs,  Comptroller,  ASD  (C3I),  ASD  (S&R), 
and  the  Director  PA&E. 

C.  THE  INITIAL  PHASE:  A  QUICK  LOOK  BY  IDA  AND  A  REVIEW  OF 
STUDIES 

The  remainder  of  this  case  study  will  focus  on  what  the  study  team  provided  the 
senior  decision  makers  and  the  issues  and  questions  that  ensued.  The  study  started  with  a 
"quick  look"  by  IDA  using  existing  tools  and  databases  that  had  been  developed  for  the 
Heavy  Bomber  Force  Study  conducted  by  IDA  for  OSD  in  1994.  This  was  followed  by  a 
review  of  the  major  Service-sponsored  weapon  mix  studies  together  with  the  IDA  quick 
look.  A  formal  “quick  look”  and  review  of  studies  is  not  typical.  Often  the  study  team 
will  perform  these  activities  for  their  own  benefit,  but  the  fonnal  reporting  and 
documenting  that  took  place  in  DAWMS  is  not  typical.  It  did,  however,  prove  to  be  very 
beneficial  to  the  data  collection  and  methodology  selection  process,  as  will  be  seen 
below.  The  studies  reviewed  were: 

•  Deep  Attack  Weapons  Mix  Study  Quick  Look,  IDA,  October  1995. 

•  Non-Nuclear  Consumables  Annual  Analysis,  (NCAA),  HQ  USAF/XOFW, 
June  1995. 

•  U.S.  Navy  (Including  Marine  Air)  Non-Nuclear  Ordnance  Requirements 
(NNOR),  Dept  of  the  Navy,  June  1995. 

•  Air  Force  Conventional  Weapons  Program  Assessment,  Air  Force  Studies  and 
Analyses  Agency,  (AFSAA),  September  1995. 
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•  Weapon  Mix  Study  for  Joint  Strike  Assessment-Revised,  Naval  Air  Warfare 
Center  -  Weapons  Division,  (NAWC-WD),  China  Lake,  October  1995. 

The  purpose  of  the  review  was  to  gain  insight  into  the  inputs,  methodologies,  and 
results  of  these  studies.  This  effort  was  a  significant  help  for  establishing  a  methodology 
and  data  collection  process  for  DAWMS.  When  there  was  a  common  approach  among 
the  studies  DAWMS  often  adopted  it.  This  allowed  the  DAWMS  study  team  to  focus  on 
those  areas  where  previous  studies  had  diverged  in  the  way  they  handled  specific  aspects 
of  the  analysis.  The  review  also  helped  identify  the  key  drivers  for  the  weapon  mix 
problem,  which,  in  turn,  helped  the  study  team  stress  the  important  aspects  of  the 
problem  in  their  research.  Finally,  it  provided  a  starting  point  for  developing  the  large 
database  that  was  ultimately  needed  by  the  study.  In  accomplishing  this  review,  the 
analysts  were  careful  to  present  their  findings  in  constructive  ways — the  review  was  not 
intended  to  be  a  critique  of  the  studies,  but  rather  a  way  to  build  upon  the  earlier  work. 

1.  The  Weapon  Types  Considered 

One  needs  to  understand  a  little  about  the  weapons  considered  in  order  to 
understand  the  results  of  the  review.  A  brief  overview  of  the  weapons  is  contained  in 
Table  1.  None  of  the  studies  incorporated  all  of  the  weapons  listed  in  Table  1. 

The  IDA  Quick  Look  study  did  not  consider  the  Army/Marine  helicopter 
launched  weapons  [Hellfire;  tube-launched,  optically  tracked,  wire-guided  missile 
(TOW)].  The  Air  Force  studies  considered  only  weapons  they  use  or  plan  to  buy  and 
therefore  excluded  ship-launched  Tomahawk  land  attack  missiles  (TLAMs),  Joint  Stand¬ 
off  Weapons  (JSOWs)  (unitary),  and  the  Standoff  Land  Attack  Missile  Enhanced 
Response  (SLAM-ER).  The  Navy  excluded  Air  Force  developed  weapons  such  as  the 
Conventional  Air-Launched  Cruise  Missile  (CALCM),  and  the  Wind-Corrected  Munition 
Dispenser  (WCMD)  system. 

2.  Summary  of  Previous  Study  Results 

Table  2  summarizes  the  results  of  the  study  review  expressed  in  terms  of  the 
aggregate  levels  of  weapons  expended.  In  the  actual  study  numerical  results  were 
provided  that  cannot  be  displayed  here  because  of  their  classification.  Instead,  the 
unclassified  description  of  the  results  is  displayed  in  Table  2  with  "High"  corresponding 
to  a  study  that  uses  the  most  of  this  type  of  munition  and  “Low”  reflecting  the  study  that 
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used  the  least  amount.  None  of  the  studies  included  helicopter  weapons.  There  is 
considerable  variation  in  the  results. 


Table  1.  Weapon  Descriptions 


Weapon  Class 

Examples 

Characteristics 

Iron  Bombs 

MK-82  /  83  /  84 

BLU-109 

500  -  2000  lb.  "dumb"  bombs  used  to  attack  fixed  targets. 

Laser-Guided  Bombs 

GBU  -10/12/15/24/ 
27/28 

Maverick 

Very  accurate  500  -  2000  lb.  weapons  that  guide  the  bomb  to 
a  laser  spot.  Used  to  attack  both  fixed  and  moving  targets. 

The  GBU-28  is  a  5,000-lb.  earth-penetrating  weapon 
delivered  by  an  F-15E. 

Cluster  Munition 

CBU  -52/58/87 

MK-20 

Consists  of  canisters  of  bomblets  that  explode  upon  impact. 
Used  to  attack  personnel  and  soft  targets. 

GPS  Guidance  Kits 

JDAM3 

Kits  that  attach  to  the  iron  bombs  to  significantly  increase 
their  accuracy. 

Sensor-Fuzed  Weapon 
(SFW) 

SFWa  and  SFW  P3I 

Specially  designed  to  kill  tanks  by  sending  a  projectile 
through  the  engine. 

INS  Guidance  Kits 

Wind  Corrected  Munition 
Dispenser  (WCMD)a 

Uses  inertial  navigation  to  help  guide  canisters  containing 
cluster  munitions  or  SFWs  to  the  desired  target. 

Medium-Range  Standoff 
Munitions 

JSOW  (CEM)a 

JSOW  (SFW)a 

JSOW  (Unitary)3 

Enables  aircraft  to  deliver  cluster,  SFW,  and  unitary  (iron 
bomb)  weapons  without  penetrating  medium-range  terminal 
air  defenses. 

Long-range  Standoff 

Munitions 

SLAM-ER 

AGM-130 

AGM-142 

JASSM3 

Enables  the  aircraft  to  stand  off  beyond  long-range  terminal 
air  defenses  when  delivering  ordnance.  They  deliver  unitary 
warheads  only. 

Very-Long-Range  Weapons 

TLAM 

CALCM 

ATACMS  1  /la  /lla  /I la3 

The  launch  platform  for  these  weapons  is  not  engaged  by  any 
defenses.  The  TLAM  is  launched  by  a  ship  or  submarine,  the 
CALCM  by  a  B-52,  and  ATACMS  by  an  Army  Multiple  Launch 
Rocket  System  (MLRS).  The  TLAM  and  CALCM  primarily 
carry  unitary  warheads.  The  ATACMS  can  carry  an 
antipersonnel  weapon  (ATACMS  l/la)  or  an  antitank  munition 
called  BAT  (ATACMS  ll/lla).  The  "a"  versions  have  a  smaller 
warhead  and  longer  range. 

Helicopter  Weapons 

Hellfire  Tow 

Longbow-Hellfire3 

Hellfire  is  the  primary  antiarmor  munition  used  by  the  Army. 

The  Longbow-Hellfire  provides  an  adverse  weather  fire-and- 
forget  capability. 

a  These  weapons  were  still  in  development  at  the  time  of  the  study. 
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Table  2.  Aggregate  Levels  of  Weapon  Expenditures  in  Recent  Major  Studies 


Weapon  Class 

Study 

IDA 

Air  Force 

Navy 

NCAA 

AFSAA 

NNOR 

NAWC 

Iron  Bombs 

Moderate 

Low 

Low 

Moderate 

High 

Laser-Guided  Bombs 

Moderate 

High 

Moderate 

Low 

Moderate 

Cluster  Munitions 
(unguided  or  with 
WCMDor  JSOW) 

Moderate 

High 

High 

Low 

Moderate 

GPS  Guidance  Kits 
(JDAM) 

Low 

High 

Moderate 

Low 

Moderate 

SFW  (using  WCMD 
or  JSOW) 

High 

Moderate 

Moderate 

Moderate 

Moderate 

Long-range  Standoff 
Munitions 

High 

Moderate 

Moderate 

Low 

Low 

Very-Long-Range 

Weapons 

High 

Moderate 

Low 

High 

Moderate 

3.  Reasons  for  Differences 

The  differences  among  the  studies  were  significant.  The  IDA  Quick  Look  Study 
favored  standoff  weapons  while  the  Air  Force  studies  (NCAA  and  AFSAA)  tended  to 
favor  precision  overflight  weapons.  The  Navy  studies,  in  general,  resulted  in  fewer 
weapons  expended.  The  differences  could  be  traced  to  the  differences  in  methodologies 
and  data  used  in  each  of  the  studies. 

One  reason  for  study  differences  concerned  the  methodologies,  and,  in  particular, 
where  the  study  placed  its  priorities.  All  of  the  methodologies  used  to  select  a  preferred 
weapon  mix  included  three  key  parameters:  weapon  effectiveness  against  the  targets  in 
the  target  set,  attrition  to  the  delivery  platforms,  and  the  cost  of  the  weapons.  Each  study 
handled  these  parameters  differently.  For  example,  both  Air  Force  studies  maximized 
target  value  destroyed  for  a  given  dollar  investment  in  weapons.  The  NCAA  study 
assigned  dollar  cost  values  to  aircraft  and  attempted  to  minimize  the  cost  of  aircraft 
attrition  associated  with  killing  a  target.  The  AFSAA  study  organized  this  differently, 
using  budget  and  aircraft  attrition  limits  as  constraints  in  a  linear  program  whose 
objective  was  to  maximize  the  target  value  destroyed.  Both  Navy  studies  focused  on 
minimizing  cost  [in  terms  of  aircraft  attrition,  operations  and  maintenance  (O&M),  and 
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weapons  expended]  to  kill  the  targets.  The  IDA  Quick  Look  study  used  a  linear  program 
to  minimize  aircraft  attrition  while  requiring  certain  targets  to  be  killed  in  a  fixed  time 
period  and  within  a  limited  budget  for  new  weapons. 

A  second  reason  for  study  differences  is  associated  with  the  target  set  used  by 
each  study.  All  of  the  Service-sponsored  studies  used  the  DIA  outyear  threat  report 
together  with  the  CINC  apportionments  to  their  Service  as  the  basis  for  their  target 
requirement.  The  IDA  target  set  included  a  large  number  of  vehicles  (trucks,  armor,  etc.) 
which  drove  a  high  use  of  cluster  munitions  and  JDAMs.  The  Navy  target  set  tended  to 
be  smaller  than  the  others,  which  in  turn  supported  their  smaller  expenditure  of  weapons. 
In  addition,  the  studies  used  very  different  descriptions  of  the  targets.  For  example,  an 
airfield  had  four  times  as  many  elements  in  the  Air  Force  target  set  compared  with  the 
Navy  set.  Reconciling  the  target  set  was  a  major  requirement  for  the  study  team,  since 
the  targets  themselves  were  major  drivers  of  weapon  requirements  and  a  key  input  for  the 
next  phase  of  DAWMS. 

A  third  cause  of  study  differences  was  the  required  probability  of  damage  (PD) 
that  each  study  required  to  meet  that  study’s  predetennined  warfighting  goals.  Achieving 
a  high  PD  requires  either  a  large  number  of  less  accurate  weapons  or  a  smaller  number  of 
very  accurate  weapons.  The  Air  Force's  use  of  high  PDs  contributed  to  their  higher 
number  of  weapons  expended.  The  Army  uses  low  PDs  in  their  weapons  requirement 
process.  Setting  appropriate  PDs  was  another  area  that  received  considerable  attention  in 
DAWMS. 

A  fourth  reason  for  study  result  differences  was  the  weapon  effectiveness 
assumed  for  each  munition.  The  Expected  Number  of  Kills  per  Sortie  (EKS)  is  the 
typical  measure  of  weapon  effectiveness,  and  this  varied  by  study.  The  relatively  high 
EKS  used  by  IDA  for  the  Sensor  Fused  Weapon  (SFW)  contributed  to  that  system’s 
preference  in  the  IDA  study. 

How  each  study  handled  costs  also  led  to  differences  in  the  results.  The  IDA  and 
AFSAA  efforts  considered  only  the  costs  of  new  weapons  and  used  a  budget  constraint  to 
limit  the  expenditure  of  these  weapons.  The  two  Navy  studies  included  replacement 
costs  for  strike  and  support  aircraft  attrition  as  well  as  aircraft  Operational  and  Support 
(O&S)  as  part  of  its  total  war  cost  and  then  orchestrated  the  target  attacks  in  order  to 
minimize  the  total  war  cost.  IDA  and  the  Air  Force  studies  did  not  include  aircraft 

9 


UNCLASSIFIED 


UNCLASSIFIED 


attrition  as  a  part  of  the  cost  equation  and  instead  implemented  constraints  for  maximum 
allowable  attrition  rates  for  various  aircraft  during  the  various  phases  of  the  campaign. 

While  the  above  paragraphs  detail  the  primary  reasons  for  study  differences,  there 
were  several  other  areas  where  the  studies  differed,  including  the  specifics  of  the  Blue 
force  structure,  the  sortie  rates  used  for  different  classes  of  Blue  aircraft,  the  weapon 
loads  carried  by  each  aircraft,  the  availability  of  inventory  weapons  in  theater,  and  the 
restrictions  each  study  associated  with  weapon  usage.  This  last  category  included 
doctrinal  or  Service  planner  inputs  that  would  exclude  a  specific  weapon  from  being  used 
to  strike  a  specific  target  type,  e.g.,  certain  Air  Force  platform/weapon  mixes  were  not 
authorized  to  attack  enemy  ships. 

D.  GENERAL  GUIDELINES  FOR  THE  DEEP  ATTACK  WEAPONS  MIX 
STUDY 

The  IDA  Quick  Look  and  the  review  of  studies  provided  useful  background  for 
DAWMS.  The  objective  of  DAWMS,  as  specified  in  the  study  guidance,  was  to 
"determine  cost-effective  mixes  (quantity  and  composition)  of  air-to-surface  and  surface- 
to-surface  munitions  for  deep  attack  operations."  In  addition,  the  OSD  leadership  issued 
general  guidance  concerning  the  conduct  of  the  study.  This  guidance  included: 

•  Use  established  joint  doctrine  for  deep  attack  operations. 

•  Use  a  two-Major  Regional  Conflict  (MRC)  scenario  (northeast  Asia  and 
southwest  Asia)  for  the  study  baseline. 

•  Analyze  force  and  weapon  mixes  in  1998  and  2006. 

•  Use  the  baseline  force  structure  outlined  in  FY  1996  President's  Budget 
(1998)  and  Defense  Planning  Projection  (DPP)  (2006). 

•  Base  force  arrivals  and  beddown  on  the  Joint  Staff  Nimble  Dancer  11-2005 
Wargame. 

•  Use  forces  in  accordance  with  1996  defense  plans. 

•  Use  enemy  threats  outlined  in  the  DIA  Outyear  Threat  Report  and  Futures 
Intelligence  Program. 

•  Consider  weapons  in  current  inventory,  in  production,  and  in  development. 
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A  major  issue  that  was  resolved  at  the  first  meeting  of  the  steering  group  was  the 
definition  of  deep  attack.  Eventually,  the  study  advisors  agreed  to  the  following 
definition: 

Deep  attack  operations  encompass  the  entire  spectrum  of  joint  activity 
conducted  beyond  close  supporting  fires.  For  the  purposes  of  DA  WMS, 

“deep”  is  defined  as  beyond  40  km  (22  nmi).  However,  DAWMS  covers 
the  en  tire  battle  area  from  the  FEBA5  on  forward. 

Some  wanted  to  exclude  the  forward  battle  from  the  analysis.  This  would  require 
setting  aside  weapons  (both  inventory  and  new  procurements)  for  the  close  battle.  Which 
enemy  forces  would  be  engaged  in  the  close  battle  depended  in  some  cases  on  how  many 
of  those  forces  first  transitioned  through  the  deep  area  to  the  close  area  and  how 
successful  the  deep  attacks  were.  Likewise,  there  was  no  easy  way  to  determine  which  of 
the  new  weapons  would  be  used  in  the  close  battle  since  that  would  depend  on  the 
severity  of  the  attack  and  the  capability  of  the  Blue  forces  to  effectively  engage.  Rather 
than  draw  artificial  lines  and  not  allow  weapons  set  aside  for  one  aspect  of  the  battle  to  be 
used  in  the  other,  even  if  the  result  would  be  beneficial,  it  was  decided  that  DAWMS 
would  include  all  air-  and  surface-delivered  direct  fire  weapons  that  attacked  targets 
beyond  the  FEBA. 

E.  THE  METHODOLOGY 

Often  the  study  customer  pays  little  attention  to  the  methodology  used  to 
accomplish  the  study,  relying  on  the  expertise  of  the  analysts  to  develop  an  acceptable 
one.  Still,  the  senior  official  who  develops  some  understanding  of  how  the  results  are 
obtained  will  often  have  more  confidence  and  be  more  likely  to  use  them  in  the  right 
way.  In  this  section  we  provide  a  brief  overview  of  the  methodology  used  in  DAWMS. 

The  DAWMS  methodology  involved  the  use  of  two  major  combat  models.  The 
Tactical  Warfare  Model  (TAG WAR)6  is  a  two-sided  simulation  representing  the  entire 


5  FEBA  refers  to  Forward  Edge  of  the  Battle  Area.  Current  terminology  uses  FLOT,  Forward  Line  of 
Troops. 

6  TACWAR  is  the  primary  campaign  model  used  by  the  Joint  Staff  (JS)  and  the  CINCs.  It  was  first 
developed  in  the  1970s  by  IDA  and  has  been  extensively  modified  over  the  years,  especially  in  the  area 
of  chemical  warfare. 
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campaign  for  both  Red  and  Blue  air-to-ground  attacks,  air-to-air  engagements,  and  the 
ground  combat  force  interactions.  Its  key  measures  of  merit  are  movement  of  the  FEBA 
and  combat  forces  attrited.  The  Weapon  Optimization  and  Resource  Requirements 
Model  (WORRM)7  is  a  one-sided  model  representing  an  attack  of  Blue  aircraft  and 
standoff  missiles  against  a  Red  ground-based  target  set  and  defenses.  It  uses  an 
optimization  technique  called  linear  programming  to  allocate  Blue  platfonns  such  as 
aircraft,  ships,  and  MLRS  and  the  weapons  they  fire  against  Red  targets  to  achieve  a 
user-defined  objective. 

The  objectives  used  in  DAWMS  were  initially  to  minimize  aircraft  and 
helicopters  attrition  [no  attrition  was  assumed  for  ships  and  Multiple  Launch  Rocket 
Systems  (MLRSs)],  and  then  to  maximize  target  value  destroyed.  The  overall  approach 
was  to  use  TACWAR  to  set  scenario  conditions  such  as  suppression  of  enemy  air 
defenses  (SEAD)  drawdown  rates,  aircraft  attrited,  FEBA  movement,  etc.,  which  could 
then  be  applied  in  WORRM  to  generate  cost-effective  weapons  mixes  for  the  U.S.  forces. 
WORRM  is  described  in  greater  detail  next  so  that  the  reader  may  more  fully  understand 
the  later  results. 

1.  WORRM  Description 

WORRM  uses  range  bands  on  the  Red  side  of  the  FEBA.  These  range  bands  can 
be  selected  to  correspond  to  various  aircraft  and  ATACMS  tactical  ranges.  Each  target 
is  in  a  range  band  and  target  kill  goals  may  be  specified  by  range  band.  Aircraft  attrition 
depends  on  the  Red  defenses  assumed  to  be  in  the  range  bands  that  the  attacking  aircraft 
must  pass  through  as  well  as  the  defenses  associated  with  the  target  itself.  WORRM  does 
not  include  Red  air  attacks  against  Blue  forces  or  air-to-air  combat,  nor  does  it  model 
ground  combat  and  artillery. 

The  campaign  consists  of  five  phases,  each  consisting  of  a  specified  number  of 

o 

days:  early  halt,  late  halt,  build  and  pound,  counter-offensive  early,  and  counter- 


7  WORRM  was  initially  formulated  to  support  the  Bottom  Up  Review  (BUR)  in  1991-92  and  has 
subsequently  been  used  in  several  major  studies. 

The  build  and  pound  phase  is  also  called  the  buildup  phase.  During  this  phase  there  is  almost  no 
ground  combat  as  the  focus  is  on  preparing  for  the  counteroffensive  by  attriting  the  enemy  forces  using 
primarily  aircraft  delivered  weapons. 
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offensive  late.9  Force  arrivals  and  target  kill  goals  vary  according  to  these  phases  while 
weather  patterns  can  also  change  in  each  phase.  In  addition,  there  was  a  fixed  time 
separation  between  the  start  of  combat  in  the  first  and  second  theaters. 

The  linear  programming  formulation10  of  WORRM  either  minimizes  aircraft 
attrition  or  maximizes  target  value  destroyed  subject  to  the  following  constraints: 

•  Constraint  1:  Number  of  targets  killed  cannot  exceed  total  number  of  targets 
available  (although  the  model  does  provide  for  some  target  regeneration). 

•  Constraint  2:  A  specified  number  of  targets  by  type  must  be  killed  in  each 
time  period  and  range  band. 

•  Constraint  3:  Aircraft  attrition  by  aircraft  type  will  be  limited  to  a  specific 
percentage  to  ensure  that  no  one  aircraft  type  suffers  significantly  more  than 
its  share  of  attrition. 

•  Constraint  4:  The  budget  that  can  be  used  to  buy  weapons  is  limited. 

•  Constraint  5:  The  number  of  weapons  used  by  type  cannot  exceed  the 
existing  inventory  (defined  as  available  inventory  plus  additional  systems 
purchased  as  of  the  time  of  the  study  in  1996). 

Over  one  million  variables  were  used  in  the  DAWMS  formulation  of  WORRM 
with  3 1  platform  types,  62  weapon  types,  and  205  target  types  represented  in  the  model. 
When  provided  with  a  weapons  budget  and  weapon  procurement  alternatives  with  their 
costs,  WORRM  will  compute  a  preferred  weapon  mix.  The  weapons  budget  was  $10.6 
billion  over  the  roughly  10-year  look  into  the  future.  To  preclude  dividing  the  budget 
between  theaters,  as  would  be  needed  if  the  optimization  were  done  sequentially,  the 
oversight  groups  directed  the  weapon  buy  to  be  optimized  across  both  theaters.  Several 
major  enhancements  were  made  to  WORRM  during  the  course  of  the  study.  Two  of  the 
most  significant  added  multiple  weather  states  and  enhanced  the  treatment  of  C4ISR. 
Both  will  be  discussed  further  in  a  later  section. 


9  Later  the  two  counter-offensive  phases  were  combined  and  a  separate  SEAD  phase  was  added  before 
the  early  halt. 

10  A  detailed  mathematical  description  of  the  model  appears  in  IDA  Document  D-2360,  A  Description  of 
the  Weapon  Optimization  and  Resource  Requirements  Model,  August  1999,  UNCLASSIFIED. 
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WORRM  produces  a  rich  set  of  outputs.  With  regard  to  weapons  it  provides  the 
number  purchased  and  the  amount  of  already  available  inventory  weapons  used  in  the 
two-MRC  conflict.  It  identifies  the  targets  they  struck,  the  weather  state,  the  platform 
type  that  delivered  them,  and,  for  aircraft,  the  release  altitude.  WORRM  post-processors 
also  calculate  the  target  value  destroyed  by  target  type  and  display  the  amount  of  target 
regeneration  that  occurred.  Finally,  for  each  aircraft  type  it  calculates  the  attrition  and 
sorties  flown  by  weather  state,  delivery  altitude,  depth  of  penetration,  weapon  standoff 
range,  and  target  types  struck. 

An  optimized  force  allocator11  like  WORRM  can  be  used  to  provide  inputs  to 
two-sided  campaign  models  like  TACWAR.  For  example,  WORRM  can  generate  the 
allocation  of  aircraft  to  missions,  an  important  input  that  defines  how  aircraft  are  used  in 
models  like  TACWAR. 

2,  Modeling  Process 

TACWAR  was  used  to  provide  a  "realistic"  overview  of  a  two-sided  campaign 
and  WORRM  was  then  implemented  to  generate  a  suggested  weapon  buy.  As  such,  it 
was  important  that  these  two  models  work  effectively  together.  This  was  done  using  a 
three-stage  process. 

Stage  1:  Use  TACWAR  and  the  current  planned  weapon  program  for  2006  to 
fight  the  campaign  in  the  two  MRCs.  From  the  TACWAR  results,  four  key  outputs  are 
passed  to  WORRM: 

•  Timing  for  the  different  phases  of  the  conflict 

•  Blue  ground  force  (except  helicopter  and  ATACM)  kills  of  Red  targets  (used 
to  reduce  kill  requirement  in  WORRM) 


11  There  are  several  other  models  that  have  structures  similar  to  WORRM.  The  Combat  Forces 
Assessment  Model  (CFAM)  is  used  by  the  Air  Force  in  weapon  and  aircraft  studies.  The  Conventional 
Targeting  Effectiveness  Model  (CTEM)  has  been  used  by  PA&E  and  J-8.  All  of  these  models  are 
based  on  a  linear  objective  function  and  linear  expressions  of  constraints.  WORRM  differs  from  these 
models  in  many  ways,  especially  in  its  in-depth  treatment  of  C4ISR  and  aircraft  attrition.  An  earlier 
weapon  optimization  code  used  a  quadratic  expression  in  the  objective  function.  It  did  not  provide  any 
additional  insights  and  the  solver  was  slower  with  more  limited  dimensions  than  comparable  LP 
solvers. 
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•  Blue  and  Red  air-to-air  combat  kills  (reduces  Blue  forces  available  and  Red 
aircraft  targets) 

•  Blue  aircraft  attrition  to  Red  surface  defenses  (used  to  adjust  the  aircraft 
attrition  rates  in  WORRM). 

Stage  2:  Use  WORRM,  with  adjusted  inputs,  to  determine  a  preferred  weapons 

buy  for  a  fixed  budget.  This  was  done  using  a  two-run  sequence.  In  Run  1,  WORRM 

determined  the  minimum  number  of  aircraft  lost  subject  to  staying  within  the  weapons 

budget  and  satisfying  the  CINC  target  kill  goals.  Target  value  was  not  used,  and  no 

12 

aircraft  type  is  permitted  to  lose  a  disproportionate  share. 

Run  2  maximizes  target  value  destroyed  using  a  bound  on  aircraft  lost  that  is  25 
percent  higher  than  the  minimum  established  in  Run  1.  For  example,  if  the  minimum 
attrition  run  lost  200  aircraft,  then  the  maximize  target  value  run  could  lose  250.  This 
second  run  was  done  to  provide  a  margin  to  protect  against  CINC  target  kill  goals  that 
were  too  low. 

Stage  3:  Rerun  TACWAR  with  this  new  weapons  mix  to  ensure  that  it  makes 
operational  sense. 

Figure  2  summarizes  the  process. 


Mathematically,  the  linear  program  minimized  (A+B)  subject  to  attrition  constraints  for  each  aircraft 
type  of  the  following  form: 

E  [platform  attrition  for  aircraft  typej]  <  A  (average  number  of  type  j  a/c  in  NEA) 

E  [platform  attrition  for  aircraft  typejj  <  B  (average  number  of  type  j  a/c  in  SWA) 
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Figure  2.  Overview  of  the  Analysis  Process 
F.  INPUT  SOURCES 

It  is  critical  for  the  study  team  to  develop  accurate,  dependable  inputs.  When 
studies  prove  to  be  invalid,  it  is  usually  the  inputs  that  are  to  blame.  It  is  critical  that  the 
study’s  customers  develop  confidence  that  the  inputs  used  are  both  technically  and 
operationally  correct.  More  than  9  months  of  the  2-year  study  were  spent  developing 
inputs  for  the  models,  mostly  for  WORRM  since  it  was  the  primary  tool  used  to 
recommend  a  weapon  mix.  When  the  same  types  of  data  were  needed  by  both  models 
(e.g.,  order  of  battle,  weapon  effectiveness,  etc.)  either  the  same  data  or  consistent  data 
were  used.14 


13  TACWAR  runs  were  done  by  the  Joint  Staff  (J-8).  Their  database  was  an  updated  version  of  the  one 
used  in  the  just  completed  Nimble  Dancer  II.  WORRM  runs  were  made  by  IDA. 

14  For  example,  WORRM  with  its  "pure"  weapon  loads  (one  type  of  munition  per  sortie)  uses  a  different 
weapon  effectiveness  input  than  TACWAR  with  its  average  weapons  load.  Consistency  is  maintained 
by  ensuring  that  both  inputs  came  from  the  same  basic  source. 
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1.  Scenario  Characteristics 

As  mentioned  earlier,  the  study  was  directed  to  use  a  two  nearly  simultaneous 
Major  Regional  Contingency  (MRC)  scenario.15  The  first  MRC  addresses  an  attack 
against  allied  forces  in  Northeast  Asia.  The  environment  was  characterized  by 
mountainous  and  forested  terrain,  poor  weather,  many  hard  and  deeply  buried  targets,  a 
relatively  unsophisticated  air  defense  threat,  large  CINC  target  goals,  a  significant  U.S. 
presence  prior  to  hostilities,  and  a  large  allied  force  to  help  in  the  warfight  (especially  in 
the  ground  war).  On  the  other  hand,  the  Southwest  Asia  MRC  involved  an  attack  by  a 
hostile  power  against  friends  in  the  region  and  was  characterized  by  desert  terrain,  good 
weather,  mobile  ground  forces,  a  modernized  air  defense,  medium  CINC  target  goals,  a 
minimal  U.S.  presence,  and  a  small  allied  military. 

2.  Delivery  Platforms 

Table  3  lists  the  launch  platforms  used  in  the  study  by  Service.  The  table  includes 
fixed  and  rotary  wing  aircraft,  MLRS,  and  naval  sea  forces.  Each  of  the  aircraft  possess 
the  following  characteristics:16 

•  Time  of  arrival  in  theater 

•  Number  of  sorties  per  day  (surge  and  sustain) 

•  Weapon  load  that  varies  by  weapon  type  and  range  of  the  target 

•  Maximum  depth  of  penetration  (helicopter  targets  can  only  strike  moderately 
deep  targets). 


15  There  was  some  early  concern  that  these  2  MRCs  were  atypical  in  that  they: 

-  Both  had  large  in-place  infrastructures  for  our  forces. 

-  Neither  had  a  significant  air  threat. 

-  Neither  involved  a  significant  naval  threat. 

In  the  end  no  additional  scenario  was  analyzed. 

16  Several  delivery  platforms  could  perform  more  than  just  strike  missions.  For  example,  the  F-16  and 
F-18  can  also  be  used  as  air-to-air  fighters.  These  multimission  aircraft  were  allocated  to  their  various 
missions  and  only  the  strike  allocation  was  used  in  WORRM. 


17 


UNCLASSIFIED 


UNCLASSIFIED 


Table  3.  DAWMS  Aircraft  and  Other  Launch  Platforms 


Service 

Launch  Platforms 

Army 

AH-1  Cobra,  AH-64  Apache,  RAH-66  Comanche 

Multiple  Launcher  Rocket  System  (MLRS) 

Navy 

F-14,  F/A-18  C/D,  F/A-18  E/F 

Submarines,  Surface  Combatants 

Air  Force 

B-1B,  B-52H,  B-2 

A-10,  F-15,  F-16,  F-22,  F-117 

Marine  Corps 

AH-1W 

AV-8B  Harrier,  F/A-18  A/C/D 

The  submarines  and  surface  combatants  are  the  launch  platfonns  for  TLAMS. 
The  MLRS  launches  the  various  versions  of  ATACMS. 

The  arrival  rates  were  provided  by  the  Joint  Staff  and  are  similar  to  those  used  in 
earlier  studies.  They  were  adjusted  to  reflect  the  lift  projected  to  be  available  in  2006. 
Each  Service  felt  that  the  warfight  would  go  better  if  their  Service  was  allocated  a  larger 
portion  of  the  airlift  available  to  allow  them  to  arrive  earlier.  Despite  these  arguments, 
the  Joint  Staff  did  not  reallocate  any  lift.  The  sortie  rates  and  weapon  loads  were 
provided  by  the  Services  for  their  systems.  Considerable  time  was  devoted  to  ensuring 
that  all  Service  inputs  were  developed  using  consistent  assumptions. 

3.  Cost 

Considerable  effort  was  expended  to  estimate  the  cost  of  each  new  weapon  [e.g., 
Joint  Air-to-Surface  Standoff  Missile  (JASSM),  SLAM-ER,  JSOW]  and  of  each  existing 
weapon  with  an  active  production  line  (e.g.,  GBU-24,  Hellfire  II,  TLAM).  The  life  cycle 
costs  included  RDT&E,  procurement  (based  on  the  learning  curve  for  the  planned  buy), 
platform  integration,  and  O&S.  All  costs  were  estimated  in  FY96  dollars  with  all  prior 
year  costs  considered  sunk.  Thus,  there  was  no  cost  associated  with  using  weapons 
already  in  the  inventory.  A  5-percent  discount  rate  was  applied  to  costs  over  the  25-year 
period  from  FY97-FY21.  Initial  plans  called  for  multiple  WORRM  runs  with  each  new 
run  reflecting  the  learning  curve  associated  with  the  recommended  procurement  in  the 
previous  run.  Early  tests  of  this  procedure  proved  cumbersome  and  time  consuming,  and 
the  results  showed  little  change  from  run  to  run.  Since  the  emphasis  of  DAWMS  was  on 
identifying  general  trends  and  preferences  for  weapon  procurement  and  not  on  providing 
a  point  estimate,  this  approach  was  abandoned. 
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4.  Weapon  Budget 

All  FY96  and  prior  year  costs  were  considered  sunk  when  determining  the  budget 
available  to  buy  additional  weapons.  The  budget  was  calculated  by  first  determining  the 
programmed  weapon  purchases  in  the  FY97  President's  Budget  for  the  next  10  years. 
The  total  planned  buy  was  then  multiplied  by  the  estimated  unit  cost  to  obtain  the  weapon 
budget.  The  FY06  budget  reflected  planned  weapon  procurement  and  deliveries  through 
the  end  of  FY06.  It  was  adjusted  to  account  for  withholds,  testing,  and  training.  The 
"combat"  weapons  budget  used  in  WORRM  for  2006  was  $10.6  billion. 

5.  Weapon  Inventory 

It  was  surprisingly  difficult  to  determine  how  many  weapons  of  each  type  the 
Services  maintained  in  their  current  inventories.  It  is  a  number  that  is  constantly 
changing  as  new  production  becomes  available  and  expenditures  (usually  for  training) 
occur.  An  estimate  was  made  of  the  inventory  at  the  end  of  FY96  and  a  percentage  of 
that  inventory,  by  weapon  type,  was  made  available  to  fight  the  two-MRC  conflicts.  This 
percentage  varied  by  weapon  type  and  was  an  input  provided  by  the  Services.  The 
weapons  were  assumed  to  arrive  in  theater  with  the  platforms  that  would  deliver  them. 
This  assumption  was  necessary  because  the  weapon  composition  depended  on  the 
WORRM  "buy"  decisions.  The  flow  of  forces  into  theater  does  include  the  weapons  in 
the  programmed  force.  Basically,  the  study  assumed  that  the  flow  would  be  adjusted  to 
reflect  the  mix  of  weapons  purchased  and  that  no  additional  expenditure  of  lift  resources 
would  be  needed. 

6.  Exclusions 

In  an  effort  to  add  doctrinal  constraints  and  operational  realism  to  the  calculations 
each  of  the  Services  provided  exclusions  to  the  study  team.  An  exclusion  prohibits  a 
specific  weapon  or  platform  from  attacking  a  specific  target.  For  example,  as  previously 
noted,  Air  Force  platforms  were  excluded  from  attacking  deployed  Red  Navy  forces 
(making  this  a  Blue  Navy  target)  and  there  was  a  prohibition  against  using  laser-guided 
bombs  (LGBs)  in  bad  weather. 
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7.  Targets 

The  target  set  (types,  numbers,  and  locations)  was  provided  by  the  JS/J8. 
Excluded  from  the  target  set  were  personnel,  mortars,  and  submarines.  The  JS,  using 
inputs  from  the  CINCs,  provided  the  time-phased  target  kill  goals.17  Some  of  these  goals 
were  specified  by  location  (e.g.,  kill  5  in  range  band  1  in  the  early  halt  phase)  and  others 
did  not  specify  location,  thus  letting  the  LP  determine  where  to  kill  the  target.  They  were 
adjusted  to  ensure  that  the  number  of  kills  required  by  the  SEAD  campaign  were 
achieved.  Finally,  the  targets  were  permitted  to  regenerate  at  rates  detennined  by  the 
Joint  Staff  in  collaboration  with  the  intelligence  community.  About  200  target  types  were 
considered  in  the  study,  ranging  from  mobile  targets  such  as  tanks  to  fixed  targets  such  as 
production  facilities  and  hardened  command  bunkers. 

8.  Weather 

Historical  records  suggest  the  two  scenarios  spanned  the  gamut  with  regard  to 
weather.  In  Northeast  Asia  (NEA)  the  weather  was  often  bad,  restricting  the  use  of  some 
weapons  and  decreasing  aircraft  availability.  Southwest  Asia  (SWA),  on  the  other  hand, 
had  mostly  good  weather.  Each  phase  of  the  war  had  a  percentage  of  bad  weather 
determined  from  the  historical  weather  characteristics  in  their  area  of  operations.  During 
periods  of  bad  weather,  some  targets  were  designated  as  "must  strikes,"  (e.g.,  tanks  near 
the  FLOT)  while  other  strikes  could  be  delayed  until  the  weather  improved  (e.g., 
production  facilities,  airbases),  provided  enough  sorties  were  available  (i.e.,  kill 
requirements  for  these  targets  were  given  weather  flexibility).  The  "must  strike"  targets 
tended  to  drive  the  procurement  of  all-weather  weapons  like  those  that  use  GPS  for 
guidance. 

9.  Weapon  Effectiveness 

Each  aircraft  sortie  with  a  specified  payload  and  each  surface-launched  missile 
(ATACMS,  TLAM)  had  an  expected  number  of  kills  associated  with  each  target.  For 
example,  when  an  aircraft  sortie  with  four  Joint  Direct  Attack  Munitions  (JDAMs) 


17  Late  in  the  study  there  was  an  effort  that  compared  these  goals  with  some  derived  using  "Effects 
Based"  or  "nodal  targeting"  criteria. 
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attacked  a  complex  of  buildings  it  was  expected  to  destroy  a  predetennined  number  of 
buildings.  This  value,  the  EKS,  depends  on  a  large  number  of  parameters  and  was  a 
major  factor  in  detennining  the  weapon  mix.  The  steps  involved  in  developing  this  input 
are  summarized  below: 

•  Specify  general  target  (e.g.,  airfield)  (as  defined  by  CINCs). 

•  Define  target  components  (e.g.,  hangars,  runway,  revetments,  ammunition 
storage  facility,  operations  building). 

•  Associate  each  target  with  a  target  on  the  standardized  target  set  used  by 
weaponeers  (this  list  contains  about  200  targets). 

•  Use  weapon  delivery  characteristics  [e.g.,  Circular  Error  Probable  (CEP), 

impact  angle]  and  the  required  PD  to  determine  the  number  of  weapons  by 

1 8 

type  needed  to  kill  each  target  type. 

•  Use  aircraft  weapon  loads  and  number  of  kills  per  weapon  to  estimate  the 
EKS.  These  values  will  vary  by  aircraft  delivery  profile  and  weather. 
DAWMS  selected  the  best  delivery  profile  for  each  of  three  altitudes  (low, 
medium,  and  high)  and  two  weather  states  (bad  and  good).  Even  after 
aggregating  the  200  targets  into  45  classes  with  similar  weapon  effects,  the 
resulting  input  table  was,  by  far,  the  largest  in  WORRM. 

•  As  appropriate,  adjust  the  EKS  for  other  factors  (e.g.,  the  EKS  for  a  standoff 
weapon  attacking  a  moving  target  may  be  adjusted  downward  as  compared  to 
the  same  weapon  being  delivered  via  overflight). 

10.  Target  Values 

As  mentioned  earlier,  the  CINCs  establish  the  minimum  number  of  targets  that 
must  be  killed  over  time  to  achieve  their  warfight  goals  (halt  the  enemy,  prepare  for  the 
counteroffensive,  etc).  Often  forces  and  weapons  are  available  to  strike  and  kill  targets 
beyond  these  minimum  goals.  When  this  occurs,  target  values  (i.e.,  the  value  of  killing  a 
target)  are  used  to  help  decide  which  targets  to  attack.  When  target  kill  goals  are  not 


ls  A  special  working  group  chaired  by  J8  developed  the  required  PDs.  They  were  a  compromise  between 
the  low  values  used  by  the  Army  and  the  high  values  used  by  the  Air  Force  and  Navy. 
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specified  by  range  band,  target  value  is  one  of  the  inputs,  along  with  aircraft  attrition 
rates  and  weapon  effectiveness/costs  used  by  the  model  to  determine  where  to  strike  the 
targets.  These  target  values  were  developed  using  a  marginal  utility  scheme  with  the 
inputs  coming  from  senior  military  warfighters  from  the  four  Services.  They  vary  by 
theater  (NEA  or  SWA),  phase  of  the  war  (halt  to  counteroffensive)  and,  in  some  cases, 
location  (the  range  band  containing  the  target). 

11.  Weapon  Standoff  Ranges 

The  standoff  range  for  air-delivered  weapons  influenced  the  attrition  rates  for 
their  delivery  platforms.  The  longer  the  standoff  range  the  fewer  the  defenses 
encountered  by  the  aircraft,  and,  usually,  the  more  expensive  the  weapon.  Standoff 
ranges  modeled  were: 

•  None  (overflight — JDAM,  LGBs) 

•  Medium-range,  about  40  nmi  (JSOW) 

•  Long-range,  about  100  nmi  (JASSM,  SLAM-ER) 

•  Outside  defenses  (CALCM,  ATACMS). 

12.  The  Threat  and  Attrition  Rates 

The  threat  to  Blue  aircraft  consisted  of  aircraft,  surface-to-air  missiles  (SAMs), 
and  anti-aircraft  artillery  (AAA).19  The  numbers,  locations,  and  system  characteristics 
were  provided  by  the  intelligence  community.  A  major  effort  was  undertaken  to  estimate 
the  effectiveness  of  each  Red  threat  against  each  Blue  aircraft.  Especially  controversial 
were  the  appropriate  degrade  factors  for  Blue  electronic  countenneasures  (ECM)  and 
maneuver."  A  separate  mission  planning  (threat  avoidance)  model  was  developed  to 
estimate  the  number  of  encounters  a  Blue  aircraft  would  have  with  Red  threats  when 


19  T  AC  WAR  did  include  air-to-air  engagements,  and  adjustments  were  made  to  inventory  levels  in 
WORRM  based  on  the  results. 

20  More  details  are  available  in  the  study's  classified  report.  Department  of  Defense  Deep  Attack  Weapons 
Mix  Study  (DAWMS),  JS/J8,  October  1997.  Several  recent  studies,  e.g.,  Analysis  of  Alternatives  for  the 
Joint  Strike  Fighter,  IDA  Report  R-407,  August  2001,  had  developed  estimates  of  these  degrade 
factors.  These  studies  were  in  general  agreement.  The  issue  concerned  how  to  combine  maneuver 
effects  with  the  on-board  ECM. 
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attacking  a  specific  target.  This  depended  on  the  standoff  range  of  the  weapon. 
Platforms  delivering  long-range  standoff  weapons  would  encounter  fewer  threats  than 
those  delivering  overflight  weapons.  The  attrition  rates  changed  by  phase  to  reflect  the 
results  of  the  SEAD  campaign.  These  data,  with  adjustments  (calibration)  based  on 
TACWAR  results,  were  used  to  populate  an  attrition  table  in  WORRM.  The 
SAM/AAA  threats  were  assumed  to  have  no  capability  against  Stealth-  aircraft  (F-117 
and  B-2)  and  standoff  weapons  (JSOW,  JASSM,  ATACMS,  TLAM,  SLAM-ER,  or 
CALCM)  after  they  were  released.  Thus,  the  B-2  and  F-117  could  overfly  targets  with 
impunity  and  hence,  had  no  reason  to  carry  the  more  costly  standoff  weapons  to  enhance 
their  survivability. 

13.  C4ISR 

Many  felt  that  C4ISR  had  been  inadequately  treated  or  not  treated  at  all  (perfect 
C4ISR)  in  earlier  weapon  mix  analyses.  As  a  result,  C4ISR  received  considerable 
emphasis  throughout  the  study.  A  C4ISR  team  composed  of  Service,  IDA,  JS,  and  OSD 
representatives  met  at  least  weekly  during  most  of  the  study.  The  team  provided  a 
methodology  and  data  (based  on  expert  judgment)  that  accounted  for  the  effects  of 
C4ISR  in  both  major  models  (TACWAR  and  WORRM).  The  methodology  considered 
the  target  acquisition,  battle  management,  weapon  delivery,  and  battle  damage 
assessment  (BDA)  elements  of  C4ISR.  More  specifically,  the  approach  considered  the 
following  factors  for  targets: 

•  Target  intelligence  quality 

•  Target  acquisition/identification  probability  (off-board  cueing  system) 

•  Target  location  accuracy 

•  Target  velocity  accuracy 


21  Some  argued  that  the  better  U.S.  training  should  be  reflected  in  the  relative  capabilities  of  the  forces. 
This  would  apply  mostly  to  the  two-sided  combat  representation  in  TACWAR.  There  were  no  special 
adjustments  made  to  WORRM  inputs. 

2~  There  was  little  change  in  the  2006  results.  This  assumption  would  not  be  acceptable  in  2014,  when  a 
larger  portion  of  the  force  would  be  Stealth  due  to  the  addition  of  significant  numbers  of  Joint  Strike 
Fighters  (JSFs). 
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•  Sensor-to-shooter  time  delay 

•  Sensor-to-shooter  information  flow  throughput 

•  Target  acquisition/identification  probability  (onboard  weapon  platform) 

•  BDA  quality  (includes  accuracy  and  timeliness). 

There  are  three  target  motion  categories  (fixed,  dwell,  and  moving)  with 
dimension  subcategories  as  indicated  in  Figure  3. 
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Figure  3.  C4ISR  Target  Categories 


Two  parameters  were  used  to  capture  the  effects  of  C4ISR: 

•  Target  Not  Found  (TNF).  This  parameter  results  in  a  sortie  aborting  its 
mission  and  returning  to  base  with  its  ordnance.  This  occurs  when  the 
mission  fails  to  find  both  its  primary  and  secondary  targets.  It  is  a  function  of 
the  cueing  the  aircraft  receives  from  off-board  sensors  during  the  mission  and 


23  TNF  is  used  in  WORRM.  In  TACWAR  the  corresponding  parameter  is  probability  of  detecting  the 
target. 
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the  ability  of  the  aircraft's  on-board  sensors  to  detect  a  target,  with  and  without 
the  cueing. 

•  Expected  Kill  per  Sortie  (EKS)  Degrade.  When  the  target  is  found  and 
weapons  are  expended  they  may  not  have  their  planned  effectiveness.  This  is 
represented  by  a  degrade  to  the  EKS.  Three  factors  contribute  to  this  degrade: 

-  BDA.  The  weapon  may  be  attacking  a  target  that  has  already  been 
killed  but  BDA  has  misclassified  as  alive. 

-  Target  Location  Error.  This  reflects  either  inaccurate  coordinates 
for  GPS  weapons  or  improperly  aimed  standoff  weapons  due  to 
unexpected  changes  in  target  movements. 

-  Target  Information  Quality.  This  may  result  in  a  degrade  if  a  target 
other  than  the  intended  target  was  found  and  attacked.  In  this  case,  the 
weapon  may  not  be  appropriate  (e.g.,  cluster  bombs  are  effective 
against  trucks  but  not  against  tanks). 

The  two  major  factors  defined  above,  TNF  and  the  EKS  degrades,  vary 
according  to  the  categories  displayed  in  Figure  4.  There  was  a  separate  numerical  value 
for  each  combination  of  categories  in  the  table  (e.g.,  a  platform  with  radar  plus  visual 
sensors  delivering  an  unguided  unitary  weapon  from  high  altitude  to  attack  a  large  fixed 
target  in  the  early  halt  in  good  weather  in  the  first  range  band  will  have  a  unique  factor). 
Long-range  standoff  weapons  like  TLAM  and  ATACM  do  not  use  sensors  onboard  their 
launch  platfonns  to  enhance  their  accuracy  and  hence,  ships  and  the  MLRS  are  not  listed 
under  the  platform  category.  These  weapon  systems  do,  of  course,  use  national  assets  to 
help  with  their  targeting. 
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Figure  4.  C4ISR  Categories 
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II.  RESULTS 


It  is  important  that  the  study  sponsors  monitor  the  study  throughout.  This 
includes  early  briefings  on  methodology  and  inputs  followed  by  briefings  of  emerging 
results.  It  is  through  this  process  that  mutual  trust  is  built.  This  will  help  the  senior 
officials  challenge  their  intuition  when  study  results  disagree  with  it.  This  may  result  in 
either  a  key  insight  or  in  a  change  to  a  questionable  input.  In  this  section  we  present  the 
results  as  they  evolved  over  the  course  of  the  study. 

A.  PRELIMINARY  RESULTS 

Early  Senior  Advisory  Group  (SAG)  briefings  were  progress  reports.  For 
example,  the  May  1996  briefing  (about  10  months  after  the  start  of  the  study)  focused  on 
two  areas: 

•  Status  of  the  Models,  WORRM  in  Particular.  Initially,  the  Linear  Program 
could  not  achieve  feasible  solutions  due  to  an  inability  to  kill  very  hard  targets 
early  (the  only  weapon  suitable  for  the  mission  was  the  Air  Force's  5,000- 
pound  bomb  (GBU-28)  and  the  needed  delivery  platforms,  F-15Es,  had  not 
yet  arrived  in  sufficient  quantity  in  theater)  and  the  difficulty  of  killing  targets 
in  bad  weather.  To  correct  these  problems  more  5,000-pound  bomb  carriers 
were  deployed  early  and  the  requirements  to  attack  some  targets  in  bad 
weather  were  relaxed  (weather  flexibility). 

•  Prioritizing  Sensitivities.  The  SAG  accepted  study  team  recommendation  to 
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place  the  following  priorities  on  sensitivity  analyses: 

-  High  priority  sensitivity  analyses: 
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Some  suggested  that  the  time  between  the  start  of  the  two  MRCs  be  decreased.  This  would  require 
reflowing  forces  to  the  second  theater,  which  was  deemed  too  time-intensive  to  do  correctly. 
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1 .  Days  of  warning  time  prior  to  D-Day  (less  warning  time  means 
fewer  forces  available  when  fighting  starts — the  fewer  forces 
available  early,  the  lower  the  probability  of  destroying  all 
required  targets  in  a  given  period  of  time) 

2.  Threat.  There  was  considerable  debate  concerning  the  number  of 
long-range  SAMs  in  SWA — the  sensitivity  analysis  was  intended 
to  show  how  Blue  effectiveness  and  the  associated  weapons 
mixes  changed  with  different  numbers  of  Red  SAMs. 

3.  Chemical  attacks  on  airfields  and  ports  and  their  potential 
impact  on  arriving  Blue  force  flows 

4.  C4ISR  probabilities  of  detection 

5.  Weapons  budget  (increase  and  decrease) 

6.  Weapons  cost  (increase  for  selected  weapons) 

7.  Countenneasures  to  include  GPS  jamming  to  impact 
effectiveness  of  Blue  platforms  and  GPS-guided  munitions  and 
dispersing  vehicles  to  reduce  the  effectiveness  of  the  SFW 

8.  Adding  developmental  costs  (new  weapons  not  in  budget)  to 
potential  weapon  procurements.  Examples  include  the  Low  Cost 
Autonomous  Attack  System  (LOCAAS)  munition  for  attacking 
armor,  JASSM  with  a  combined  effects  bomblet  (CEB)  payload, 
a  more  accurate  JDAM,  etc. 

Medium  Priority 

1 .  Required  probability  of  damage  (impact  of  lower  PDs  to  reflect 
Army  use) 

2.  Logistics  availability 
Low  Priority 

1 .  Weather  (better  or  worse  than  baseline) 

2.  Adding  forces 


28 


UNCLASSIFIED 


UNCLASSIFIED 


3.  Swinging  more  forces  between  theaters  (in  the  baseline,  only 
some  bombers  and  F-117s  redeploy  from  the  first  theater  to  the 
second).  All  other  forces  deploying  to  the  second  theater 
originate  in  CONUS  or  Europe. 

4.  Target  values 

5.  CINC  target  kill  objectives. 

B.  DEFENSE  SCIENCE  BOARD  REVIEW 

Because  the  study  could  potentially  impact  Service  munition  budgets  as  well  as 
indirectly  measure  the  potential  success  of  various  force  applications  in  two  warfighting 
CINCs'  areas  of  responsibility,  it  received  high  level  attention  both  within  and  outside  the 
Department  of  Defense.  Both  the  White  House  and  Congress  mandated  a  review  of  the 
study  and  its  methodology  by  a  Defense  Science  Board  (DSB)  panel: 

The  Defense  Science  Board  will  be  asked  to  form  a  special  panel  to 
provide  an  independent  assessment  of  the  analytical  tests  and  models 
employed  and  make  recommendations  for  improvements  in  the  study's 
analytical  approach. 

(The  White  House,  Statement  by  Press  Secretary,  8  Feb  1996). 

The  Secretary  of  Defense  shall  establish  an  ad  hoc  review  committee 
under  the  auspices  of  the  Defense  Science  Board  to  establish  the 
methodological  approach  to  the  tradeoff  study,  to  establish  a  broad  range 
of  stressing  scenarios  of  interest,  and  to  review  assumptions  regarding  the 
analysis  to  be  conducted. 

(Section  8111,  FY  1997  DoD  Appropriations  Act) 


The  members  of  the  DAWMS  DSB  panel  were: 

•  Professor  Walter  E.  Morrow,  Chairman 

•  General  Michael  P.C.  Cams,  USAF  (Ret.) 

•  General  John  W.  Vessey,  USA  (Ret.) 

•  Admiral  Leon  A.  Edney,  USN  (Ret.) 

•  Major  General  Ray  M.  Franklin,  USMC  (Ret.) 

•  Dr.  John  D.  Christie,  Logistics  Management  Institute  (LMI) 
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•  Mr.  Robert  J.  Murray,  Center  for  Naval  Analyses  (CNA) 

•  Mr.  Michael  D.  Rich,  RAND 

•  Honorable  Harold  Brown,  Reviewer. 

The  initial  meeting  took  place  in  Irvine,  California,  as  part  of  the  DSB’s  summer 
study  series.  After  receiving  inputs  from  the  Services,  department  offices,  and  study 
team  members,  the  DSB  generally  approved  of  the  methodology  used  to  estimate  weapon 
needs  and  suggested  a  wide  range  of  sensitivities  designed  to  help  address  Service 
concerns.  In  addition,  they  recommended  that  the  schedule  be  extended  as  necessary  so 
as  not  to  sacrifice  quality  to  meet  deadlines. 

C.  INITIAL  RESULTS 
1.  Early  Trends 

In  the  fall  of  1996,  the  emerging  results  for  addressing  two  nearly  simultaneous 
Major  Regional  Conflicts  in  2006  suggested  a  significantly  different  weapons  mix  be 
pursued  than  that  in  the  existing  program.  More  specifically,  the  initial  results  suggested 
the  following: 

•  Accurate  overflight  weapons 

-  Maintain  the  JDAM  program 

-  Buy  more  LGBs  (especially  GBU-24s) 

-  Buy  more  Hellfire  IIs  and  fewer  Longbows  for  helicopters 

-  Decrease  the  WCMD  program 

•  Accurate  standoff  weapons 

-  Increase  the  JASSM  buy 

-  Increase  the  buy  of  ATACMS  Ha  [the  extended  range  brilliant  anti-tank 
(BAT)  weapons  version] 

-  Decrease  the  buy  of  TLAMs  and  JSOWs  (all  versions) 

•  Submunitions 

-  Decrease  overall  buy  of  SFWs  with  preference  for  SFW  P3I 

-  Use  existing  cluster  munitions  in  WCMD  kits. 
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2.  Service  Requests 

These  preliminary  results  suggested  the  acquisition  of  a  greater  number  of  Army 
weapons  than  programmed,  suggested  a  change  in  the  mix  of  Air  Force  weapons  toward 
more  GBU-24s  and  JASSMs  and  fewer  JSOWs,  and  provided  only  limited  support  for 
Navy  weapon  programs.  As  is  often  true  when  a  study  suggests  a  change  in  a  program,  a 
major  review  of  the  methodology  and  inputs  ensued.  The  following  are  some  of  the 
resulting  requests  and  changes  received  from  the  Services: 

•  Air  Force 

-  Add  the  Airborne  Laser  (ABL)  to  the  weapon  mix 

-  Increase  the  SAM  threat  in  early  halt 

•  Army 

-  Zero  the  target  value  from  infrastructure/strategic  targets  and  trucks 

-  Use  Anny  probability  of  damage  requirements  for  weapon  effectiveness 
calculations 

•  Navy 

-  Add  the  newly  postulated  arsenal  ship  with  500+  vertical  launch  cells  to 
the  baseline 

-  Modify  the  standoff  range  for  the  SLAM-ER  and  add  it  to  the  list  of 
weapons  eligible  for  purchase 

-  Triple  the  attrition  for  platforms  delivering  overflight  weapons  in  the  early 
halt  phase 

•  Marines 

-  Increase  the  SAM  threat  in  the  early  halt  phase. 

3.  Other  Areas  Needing  More  Analysis 

In  addition,  there  were  several  areas  that  were  unsatisfying  to  many  of  the 
DAWMS  study  team  members,  to  include: 

•  Helicopter  Attrition.  Many  felt  that  a  major  weakness  of  the  study  was  its 
inability  to  derive  helicopter  attrition  rates  in  a  manner  consistent  with  the 
aircraft  attrition  rates.  Attempts  to  estimate  encounters  with  SAMs  and  AAA 
for  deep  strike  missions  failed.  A  special  RAND  study  was  commissioned  to 
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address  this  issue,  but  results  were  not  available  in  time  to  support  DAWMS. 
In  the  absence  of  an  analytical  approach,  expert  military  judgment  was  used  to 
set  the  helicopter  attrition  rates. 

•  Uncertainty  in  Weapon  Costs.  Sensitivity  analyses  were  conducted  for  the 
costs  of  JASSM,  ATACMS  II,  JSOW,  SFW,  JDAM,  and  WCMD. 

•  Differences  in  Target  Values  Among  the  Services.  The  initial  results  used 
the  average  of  the  Service-provided  target  values.  Sensitivity  runs  were  made 
with  each  Service's  target  values  with  surprisingly  little  change  in  weapons 
purchase  recommendations. 

•  Impact  of  the  Size  of  Weapons  Budget.  Excursions  examined  a  25  and  50 
percent  higher  budget  and  a  25  percent  lower  budget. 

•  Impact  of  the  Joint  Staff  Added  "Flexibility"  Factors  to  the  Target  Kill 
Goals  Provided  by  the  CINCs.  The  low  use  of  some  of  the  standoff 
weapons  and  a  concern  that  the  CINCs  were  too  conservative  in  their 
estimates  of  required  kills  elevated  this  in  priority  relative  to  the  initial  list 
(see  paragraph  A).  Runs  were  made  with  these  smaller  CINC  goals,  resulting 
in  lower  aircraft  attrition  and  an  increase  in  the  use  of  standoff  weapons. 

D.  REVISED  RESULTS 

By  January  1997  the  results  of  the  wide  range  of  excursions  suggested  the 
previous  fall  had  been  accomplished  and  the  relevant  data  changes  had  been  made.  The 
results  were  briefed  to  the  review  groups  and  individually  to  the  four  Service  vice  chiefs. 
The  new  results  are  summarized  in  Table  4.  The  last  column  in  the  table  contains  the 
excursion  that  resulted  in  the  largest  purchase  of  each  weapon  type.  In  general,  it  was  a 
relaxing  of  conditions  (higher  budget  or  fewer  required  kills)  that  resulted  in  the  largest 
purchase.  The  new  results  did  modify  some  of  the  earlier  insights  but  not  enough  to 
satisfy  all  of  the  Services. 

Basically,  the  optimization  favored: 

•  Weapons  with  long  standoff  ranges  (JASSM,  SLAM-ER,  ATACMS  II)  in  the 
early  halt  phase  when  Red  defenses  were  most  effective 
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•  Relatively  inexpensive  accurate  unitary  weapons  (LGBs,  JDAMS)  to  attack 
fixed  relatively  hard  targets  after  Red  defenses  had  been  suppressed 

•  Hellfire  and  existing  inventory  weapons  like  TOW  and  Maverick  to  attack 
mobile  targets  [tanks,  APCs,  Infantry  Fighting  Vehicles  (IFVs),  artillery] 

•  ATACMS  II,  JDAM,  JASSM,  and  SLAM-ER  to  attack  targets  in  bad  weather. 


Table  4.  New  Results 


Suggested  Change  to  Programs 

Area 

Program 

Initial  Results 

New  Results 

Most  Favorable 
Sensitivity 

Accurate  Overflight 
Weapons 

JDAM 

Maintain 

Maintain 

Larger  budget 

LGBs 

Increase 

Increase 

Larger  budget 

Hellfire 

Increase 

Increase 

Higher  budget 

Longbow 

Decrease 

Slight  decrease 

Lower  target  goals 

WCMD  (both  OEM 
and  SFW  versions) 

Decrease 

Decrease 

Larger  budget 

Standoff  Weapons 

JASSM 

Increase 

Slight  increase 

Larger  budget 

SLAM-ER 

N/A 

Increase 

Several 

TLAM 

Decrease 

Decrease 

None 

JSOW  (OEM) 

Decrease 

Slight  decrease 

Lower  target  kill 
goals 

JSOW  (SFW  and 
Unitary) 

Decrease 

Decrease 

Lower  target  kill 
goals 

ATACMS  II  (BAT) 

Increase 

Increase 

Larger  budget 

ATACMS-IA 

(APAM) 

Decrease 

Decrease 

None 

Submunitions 

SFW 

Decrease 

Decrease 

Larger  Budget 

Those  weapons  excluded,  for  the  most  part,  were  relatively  expensive: 

•  Shorter  range  standoff  weapons  like  JSOW  and  Longbow 

•  Longer  range  standoff  weapons  like  TLAM 

•  Cluster  munition  delivery  systems  like  ATACMS  II 

•  Guidance  kits  like  WCMD. 
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E.  THE  FINAL  ROUND 

The  Services,  especially  the  Navy,  continued  to  object  to  the  "new  results."  There 
was  a  renewed  effort  to  reexamine  every  assumption  with  emphasis  placed  on 
"operational  realism."  In  addition,  new  data  arrived  on  weapon  costs  and  many  new 
sensitivities  were  suggested. 

1.  Key  Changes 

The  spring  of  1997  brought  a  number  of  key  changes: 

•  New  Cost  Data.  The  Air  Force  provided  new  cost  projections  for  the  WCMD 
and  JASSM.  These  projections  significantly  lowered  the  unit  procurement 
cost  of  both  programs.  This  new  cost  data  was  accepted  after  a  careful  review 
by  the  cost  team. 

•  Adjustments  Based  on  Battle  Staff  Recommendations  (operational 
realism  focus) 

-  Revise  the  rates  at  which  high  and  low  altitude  SAMs  are  destroyed 
through  time. 

-  Increase  the  emphasis  on  the  suppression  of  enemy  air  defenses  by 
adding  a  SEAD  phase  prior  to  the  early  halt.  In  this  phase  the  primary 
targets  would  be  air  defenses.  Adjust  subsequent  attrition  rates 
accordingly. 

-  Modify  helicopter  attrition  rates  to  reflect  survivability  differences 
among  platforms  (AH-64A,  AH-64D,  AH-1W)  and  the  weapons 
(Hellfire,  Longbow)  they  are  carrying. 

-  Increase  the  restrictions  on  helicopter  employments  into  deeper  range 
bands. 

•  Sensitivity  Analyses  Added 

-  More  scenarios,  especially  a  short  warning  scenario.  In  this  scenario 
the  U.S.  is  given  less  time  to  deploy  its  forces  before  the  war  starts. 

-  Less  emphasis  on  infrastructure  targets  by  eliminating  the  possibility 
of  attacking  these  targets  once  the  CINC  goals  have  been  met. 

34 


UNCLASSIFIED 


UNCLASSIFIED 


2.  Hybrid  Results 

The  conditions  for  the  new  set  of  runs  were  referred  to  as  the  hybrid  conditions 
and  the  results  became  known  as  “the  hybrid  results.”  Some  of  the  previous  "reduced 
buy"  weapon  systems  were  found  to  be  sensitive  to  the  scenario  assumptions.  In  other 
words,  the  study  conclusions  continued  to  evolve.  They  are  summarized  in  Table  5. 
Included  for  weapon  systems  with  modified  recommendations  from  the  previous  round  is 
a  brief  rationale  as  to  the  major  factor  that  caused  the  change. 

Several  actions  occurred  which  were  at  least  in  part  motivated  by  the  results 
displayed  in  Table  5.  For  example,  it  was  reported  that  the  Army  delayed  closing  the 
Hellfire  production  line  and  was  de-emphasizing  the  ATACMS  la,  that  the  Air  Force  was 
going  to  keep  open  the  LGB  production  line,  and  that  the  Navy  was  going  to  explore  new 
production  of  the  SLAM-ER  (over  and  above  the  SLAM  to  SLAM-ER  upgrade 
program).  In  addition,  the  DoD  had  new  insights  with  regard  to  the  vulnerabilities  of 
GPS-guided  weapons  to  jamming,  the  effect  of  decoys,  smoke  and  obscurants  on  the 
preferred  weapons,  and  which  weapons  still  under  development  showed  the  most 
promise. 

Perhaps  the  most  important  result  of  the  study  was  the  observation  that  the 
programmed  munitions  investment  budget  between  1997  and  2006  is  sufficient  to 
provide  the  next-generation  munitions  needed  to  maintain  the  U.S.  advantage  over 
potential  opponents.  This  was  further  supported  by  excursions  with  a  smaller  budget  that 
proved  to  be  unacceptable  (the  linear  program  was  not  able  to  generate  a  feasible 
solution). 

Still,  the  critiques  of  the  study  continued  as  several  Service-favored  programs  had 
not  done  as  well  as  they  expected.  Some  of  the  Services  felt  that  the  whole  concept  of 
DAWMS  infringed  upon  their  Title  10  authority  to  “organize,  train  and  equip”  their 
forces  for  the  warfight.  These  final  critiques  are  discussed  next. 
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Table  5.  Hybrid  Results 


Area 

Program 

Suggested  Change  to  Programs 

Hybrid  Results 

Initial  Results 

New  Results 

Accurate  Overflight 
Weapons 

JDAM 

Maintain 

Maintain 

Maintain 

LGBs 

Increase 

Increase 

Increase  (but  less 
than  before  due  to 
survivability  of 
platform) 

Hellfire 

Increase 

Increase 

Slight  decrease  (due 
to  survivability  of 
platform) 

Longbow 

Decrease 

Slight  decrease 

Increase  (due  to 
better  platform 
survivability) 

WCMD  (both  OEM 
and  SFW  versions) 

Decrease 

Decrease 

Increase  (due  to  cost 
decrease) 

Standoff  Weapons 

JASSM 

Increase 

Slight  increase 

Increase  (due  to  cost 
decrease) 

SLAM-ER 

N/A 

Increase 

Increase 

TLAM 

Decrease 

Decrease 

Decrease 

JSOW  (CEM)a 

Decrease 

Slight  decrease 

Decrease 

JSOW  (SFW) 

Decrease 

Decrease 

Decrease 

JSOW  (Unitary) 

Decrease 

Decrease 

Maintain  (all  weather) 

ATACMS  II  (BAT) 

Increase 

Increase 

Increase 

ATACMS-IA 

(APAM) 

Decrease 

Decrease 

Decrease 

New  Submunitions 

SFW/SFW  P3I 

Decrease 

Decrease 

Maintain  (replaces 
Hellfire) 
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III.  CRITIQUES,  CONTRIBUTIONS,  AND  LESSONS  LEARNED 

A.  CRITIQUES  OF  THE  STUDY 
1.  Service  Involvement 

During  the  course  of  the  study  the  DAWMS  study  leaders  in  J-8  and  OSD 
devoted  considerable  effort  to  keeping  each  of  the  Services  informed  and  involved.  This 
was  important  as  it  was  the  Services  that  would  need  to  implement  the  study's 
recommendations.  Examples  of  Service  participation  include: 

•  Each  Service  had  final  say  on  the  characterizations  used  as  inputs  for  their 
systems  (e.g.,  sortie  rates,  weapon  loads,  etc.). 

•  The  Services  were  included  and  encouraged  to  participate  in  all  meetings. 

•  The  Services  provided  most  of  the  key  operational  inputs 

-  Values  of  targets 

-  Pace  of  the  SEAD  campaign  (via  the  Battle  Staff) 

-  Sortie  rates  and  weapon  loadouts  for  aircraft 

-  Exclusions  in  weapon/platform  usage  (e.g.,  Air  Force  platfonns  were 
precluded  from  attacking  enemy  ships  outside  port  areas). 

•  Twice  the  Services  were  given  the  opportunity  to  specify  run  conditions  for 
sensitivities  (some  of  these  were  discussed  earlier).  The  DAWMS  study  team 
would  then  accomplish  the  run  and  display  the  results  to  the  group.  In  this 
way  each  Service  could  explore  results  using  conditions  favorable  to  that 
Service  (e.g.,  the  Army  precluded  heavy  bombers  from  close-in  range  bands; 
the  Marines  and  Navy  directed  higher  than  estimated  JASSM  costs,  etc.) 
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2.  Model  Critiques 

Still  the  study  was  severely  criticized  in  some  circles.  Some  of  these  criticisms 
focused  on  the  models. 

•  For  example,  the  following  appeared  in  a  national  magazine: 


Crazy  Analytical  Models 

At  one  point  [in  the  DAWMSJ,  it  looked  as  if  we  would  be  very  foolish  to  buy 
any  [Joint  Stand-off  Weapon],  and  we  ought  to  buy  as  much  [Wind- 
Corrected  Munitions  Dispenser]  as  we  possibly  could.  As  we  looked  at  it,  we 
said,  ‘Why  would  the  model  consistently  choose  wind-corrected  munitions 
over  the  Joint  Stand-off  Weapon?’  Finally,  we  realized  it  was  because  the 
model  wouldn't  distinguish  between  stand-off  at  altitude  and  stand-off 
horizontal.  We  had  to  go  back  in  and  say,  ‘There  simply  are  targets  that  you 
don't  want  to  fly  over.  ’  We  had  to  try  to  find  a  way  to  tell  the  model  that 
there  are  tactical  needs  for  certain  kinds  of  munitions.  We  never  did  get  the 
model  fixed. 

Source:  “The  New  View  of  Airpower,”  Maj  Gen  Charles  Link,  Air  Force 
Magazine,  August  1997. 

In  reality,  the  criticism  was  not  warranted  as  the  model  had  significantly  lower 
attrition  for  standoff  weapons  (especially  for  the  longer  range,  stand  outside  area 
defense  weapons,  like  JASSM  and  SLAM-ER). 

•  Another  model-related  critique  concerned  the  limited  number  of  weather 
states.  WORRM  could  incorporate  two,  good  and  bad  weather.  This  forced 
some  tough  choices.  Man-in-the-Loop  weapons  like  SLAM-ER  can  be 
delivered  in  weather  conditions  that  would  preclude  medium-altitude  LGB 
delivery  (e.g.,  low  clouds  would  have  little  impact  on  the  SLAM-ER).  On  the 
other  hand,  SLAM-ER  would  not  be  suitable  for  delivery  in  driving  rain  but  a 
GPS-guided  weapon  would  be  unaffected.  This  criticism  has  merit  and 
WORRM  was  upgraded  in  its  later  versions  to  include  more  weather  states. 

•  Perhaps  the  most  unusual  critique  concerned  the  WORRM  linear  program. 
The  critiques  stated  that  "Everyone  speaks  a  special  language  called 
’WORRM  Speak'  which  is  very  complex  and  harder  than  most  operators  can 
comprehend."  Linear  programs  can  be  subject  to  interesting  shifts  in 
allocation  depending  on  how  the  model’s  objective  and  constraints  are 
characterized,  but  such  changes  are  well  understood  by  the  analytic 
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community  and  usually  clearly  explainable  to  operators  and  decisionmakers. 
Perhaps  part  of  the  problem  with  explaining  these  results  was  due  to  the 
complexity  of  the  process  that  evolved  over  time  as  the  study  team  tried  to 
accommodate  a  wide  variety  of  operational  and  tactical  considerations. 

•  Some  complained  that  WORRM  had  not  been  properly  verified  and  validated 
(V&V).  Extensive  verification  was  accomplished  with  each  modification  by 
the  IDA  team.  In  results  validation  the  analyst  varies  inputs  and  checks  to  see 
if  the  results  move  in  the  right  direction  (i.e.,  make  sense).  This  validation 
was  done  using  the  750  runs  accomplished  during  the  study  and  the  many  runs 
made  when  earlier  versions  of  the  model  were  used  to  provide  results  for  the 
Major  Aircraft  Review,  the  Bottom  Up  Review,  and  the  Heavy  Bomber  Force 
Study. 

•  The  Air  Force  felt  that  the  TACWAR  model  did  not  adequately  account  for 
the  effects  of  airpower  and  its  ability  to  significantly  influence  the  movement 
of  the  FLOT,  especially  the  linkage  between  deep  attacks  and  the  ground 
battle.  In  particular,  air  strikes  against  logistics  facilities  and  infrastructure 
will  limit  resupply  of  front  line  forces  and  impact  the  ground  battle.  Other 
criticisms  of  TACWAR  included  its  inability  to  model  flanking  maneuvers 
and  airborne  assaults  behind  enemy  lines. 

3.  Critiques  of  Model  Inputs 

Most  of  the  critiques  concerned  model  inputs.  A  few  are  summarized  below  with 
the  specific  complaint,  often  voiced  in  public  meetings  and  publications  followed  by  an 
explanation  of  what  actually  was  implemented  in  the  models.  Occasionally  an  issue 
would  be  raised  in  one  of  the  review  forums  or  by  the  battle  staff  and  corrected  with 
model  changes  or  addressed  by  sensitivity  analysis  in  the  next  cycle.  Unfortunately,  the 
complaint  would  continue  to  be  repeated  for  months  after  it  was  initially  raised  by 
individuals  who  had  been  briefed  on  the  study  but  who  had  no  first-hand  knowledge  of 
the  process  or  the  evolution  of  the  results.  Some  of  the  more  persistent  complaints 
included: 

•  WORRM  did  not  have  a  SEAD  campaign.  (The  model  implemented  a  joint 
SEAD  campaign  developed  by  the  Battle  Staff.) 
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•  OH-58  Kiowa  warriors  killed  a  significant  number  of  trucks  with  rockets. 
(Rockets  were  removed  from  the  study  during  the  preliminary  phase.) 

•  Attrition  rates  were  too  low  or  too  high.  (The  relative  attrition  rates  across 
fixed-wing  aircraft  had  a  solid  analytical  basis  that  was  agreed  to  by  all  of  the 
Services.  There  was  less  support  for  the  rotary-wing  attrition  rates  and  no 
clear  analytic  basis  for  the  relative  relationship  between  fixed-  and  rotary¬ 
wing  attrition  inputs  although  these  issues  were  addressed  through  sensitivity 
analysis.) 

•  Sortie  rates  did  not  depend  on  base  location.  (In  fact,  sorties  rates  were  based 
on  averages  of  the  aircraft  at  the  specified  beddown  bases). 

•  Study  inputs  had  no  basis  in  fact.  (The  study  used  Service-derived 
characteristics  based  on  historical  data,  detailed  modeling  outputs,  and/or  the 
best  military  judgments  available.) 


B.  WHAT  MADE  DAWMS  UNIQUE 
1.  The  Problem 

And  let  it  be  noted  that  there  is  no  more  delicate  matter  to  take  in  hand,  nor 
more  dangerous  to  conduct,  nor  more  doubtful  in  its  success,  than  to  set  up 
as  a  leader  in  the  introduction  of  changes. 

Niccolo  Machiavelli  (1469-1527).  Chapter  VI,  The  Prince. 

DAWMS  was  an  attempt  to  change  the  way  DoD  looked  at  weapons 
procurement.  Every  year,  each  Service  is  given  a  part  of  the  CINC  target  set  and  asked  to 
develop  a  weapons  program  that  is  effective  against  this  allocation.  In  DAWMS  the 
analysis  of  weapon  needs  was  done  from  a  joint  perspective.  The  target  set  was  not 
partitioned  into  Service  shares.  Instead,  the  full  target  set  was  used  and  the  Service 
weapon  programs  were  judged  on  their  ability  to  service  all  the  targets.  For  the  first  time 
trade-offs  between  weapons  developed  by  the  individual  Services  were  evaluated  in 
tenns  of  their  contributions  to  overall  CINC  objectives.  ATACMS,  JASSM,  JSOW,  and 
TLAMs  were  directly  compared  with  each  other. 
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2.  Oversight 

Oversight  was  extensive  and  included  an  expanded  JROC  co-chaired  by  OSD 
A&T  and  the  Vice  Chairman,  JCS,  a  senior  steering  group  at  the  three-star  level,  multiple 
sponsors  (J-8,  Policy,  Acquisition,  PA&E),  a  DSB  methodology  review,  and  considerable 
Congressional  interest.  Oversight  officials  were  often  quick  to  challenge  results  that  did 
not  match  previous  analyses,  even  though  the  context  in  which  the  results  were  developed 
had  changed.  For  example,  results  that  were  fully  defendable  in  the  context  of  a  single 
Service  destroying  a  selected  portion  of  the  target  base  might  no  longer  show  the  same 
contributions  when  all  targets  were  available  for  all  Services  to  attack.  Often  the 
distinction  between  defending  previous  positions  and  gaining  the  greatest  insight  from 
the  new  process  was  blurred.  To  the  extent  that  the  study  helped  the  community  think 
through  U.S.  warfighting  concepts  and  their  implications  for  weapon  acquisition,  it  was  a 
significant  success. 

3.  Service  Participation 

There  was  extensive  Service  participation  at  all  levels  as  discussed  above  during 
most  of  the  study.  This  participation  took  place  at  the  oversight,  analytic,  and  Battle 
Staff  levels.  While  the  Service  participants  often  did  not  agree  on  major  elements  of  the 
study  and  therefore  drove  many  of  the  sensitivity  analyses  to  determine  how  differences 
in  inputs  and  approaches  would  impact  results,  in  the  end  the  study  orchestrated  the  most 
extensive  cross-Service  involvement  ever  encompassed  in  a  DoD  study. 

4.  Study  Visibility  and  Openness 

The  DAWMS  databases  were  available  to  all  participants  (several  shadow  studies 
were  done  by  the  Services  during  the  course  of  the  study  using  the  databases).  All 
participants  had  access  to  the  models  and  ample  opportunity  to  review  the  methodology. 
Finally,  study  results  were  briefed  at  least  weekly  to  provide  all  participants  with 
complete  access  to  emerging  results  in  a  timely  manner. 


2:1  The  exception  occurred  during  the  last  month  of  the  study  when  the  final  briefing  was  being  prepared. 
The  Services  were  excluded  by  the  sponsors  due  to  a  mandatory  deadline  imposed  by  the  Secretary  of 
Defense.  There  was  no  time  for  extensive  Service  coordination. 
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C.  ANALYTICAL  CONTRIBUTIONS 

1.  Collaborative  Study 

DAWMS  was  one  of  the  first  (if  not  the  first)  major  study  conducted  by  DoD  that 
was  truly  collaborative.  It  showed  that  Joint  Collaborative  Analysis  (JCA)  could  really 
be  done  but  at  a  price — in  time,  resources,  complexity,  number  of  cases,  etc.  Through 
this  process  each  Service  learned  about  the  tactics  and  capabilities  of  the  weapon  systems 
of  their  sister  Services. 

2.  Brought  Analysis  to  Operators  and  Senior  Managers 

DAWMS  brought  analysis  to  the  decision-makers.  The  senior  officials  really  got 
"into"  the  analysis-examining  data  input,  output,  exploring  dependencies  and  effects,  and 
initiating  new  cases.  In  some  cases,  it  was  the  first  time  some  senior  operators  were 
exposed  to  analysis,  including  what  it  can  and  cannot  do.  In  a  few  instances  operators 
were  horrified  that  the  analysis  made  so  many  assumptions  and  that  it  really  did  not 
mirror  the  "real  world."  It  was  an  important  educational  process. 

3.  Program  Impact:  The  Positive  View 

Widely  divergent  opinions  exist  as  to  the  value  of  DAWMS.  These  differences 
largely  involve  how  the  weapons  program  changed  as  a  result  of  the  study.  Those  with  a 
positive  opinion  of  DAWMS  argue  that  DAWMS  confirmed  that  our  weapons 
investment  program  is  about  right  in  terms  of  its  size  and  composition.  It  showed  the 
importance  of  the  use  of  precision  weapons  in  modern  warfare,  including  the  use  of 
JDAM,  long  before  Kosovo.  They  also  state  that  many  programs  were  affected  by  the 
attention  given  by  the  study,  resulting  in  some  adjustments  to  the  size  of  some  programs 
and  pressures  to  reduce  costs  in  others.  It  showed  the  need  for  reducing  the  cost  of 
JSOW,  something  the  Navy  undertook  in  earnest  later  in  the  program.  It  directly  affected 
the  Army's  Hellfire  II  and  ATACMS  programs.  It  exposed  the  deficiency  of  DoD's 
ability  to  fight  in  bad  weather  because  there  were  not  enough  all-weather  precision 
weapons.  Also,  DAWMS  may  have  indirectly  caused  LGB  production  lines  to  remain 
open  longer  than  had  been  scheduled.  In  summary,  DAWMS  did  recommend  a 
"balanced"  weapon  mix  that  included  all  types  of  weapons,  ranging  from  specialized 
accurate  weapons  for  striking  very  hard  targets  to  long-range  standoff  weapons  used  to 
attack  well  defended  targets.  It  highlighted  the  importance  of  weather,  C4ISR,  and  cost 
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when  deciding  on  the  preferred  weapons  mix.  It  did  not  advocate  a  point  solution,  but 
rather  it  suggested  areas  that  should  receive  either  more  or  less  emphasis. 

4.  Program  Impact:  The  Negative  View 

Those  opposed  to  the  study  argue  that  the  thousands  of  staff-hours  and  millions  of 
dollars  spent  in  conducting  DAWMS  were  ultimately  an  unnecessary  diversion  since  the 
conclusions  were  essentially  that  the  Services  are  handling  the  programs  appropriately 
and  any  cost  or  programmatic  changes  would  have  occurred  anyway. 

5.  Ancillary  Products 

Still,  both  groups  generally  agree  that  there  were  some  useful  ancillary  study 
products,  to  include: 

•  Several  fully  developed  two-MTW  scenarios  covering  the  years  from  1998  to 
2014. 

•  An  improved  deep  attack  modeling  methodology  that  linked  an  optimizer 
(WORRM)  with  a  combat  model  (TACWAR). 

•  The  initial  steps  to  integrate  C4ISR  systems  in  campaign  modeling  that  later 
spurred  on  follow-on  studies  to  further  develop  this  capability. 

•  The  large  and  detailed  DAWMS  database  that  was  blessed  by  the  Services 
and  used  in  several  important  studies  in  succeeding  years.  Key  parts  of  that 
database  include: 

-  Weapon  effectiveness  in  the  presence  of  countermeasures 

-  Mobility  and  Blue  force  flows 

-  Red  orders  of  battle  and  Concepts  of  Operations  (CONOPS) 

-  Allied  force  inputs  and  host  nation  support 

-  Unit  weapon  costs 

-  Complete  set  of  Service-derived  target  values  and  goals 

-  Estimates  of  aircraft  attrition  rates  with  countermeasures. 
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D.  LESSONS  LEARNED 

Acceptance  of  Results  Depends  on  Acceptance  of  the  Analytical  Process.  The 
DAWMS  modeling  process  was  complex.  It  involved  two  non-trivial  models,  each  with 
very  large  databases  and  each  with  significant  warfighting  simplifications  implemented  to 
allow  focus  on  broader  issues.  The  DAWMS  study  team  needed  to  do  a  better  job 
"simply"  explaining  the  process — and  they  needed  to  repeat  it  over  and  over  again,  both 
to  reinforce  changes  implemented  as  a  result  of  Service  and  other  overseer  input  and  to 
bring  new  participants  from  the  various  organizations  up  to  speed  on  changes  and  new 
results. 

Keep  the  Results  in  Context.  The  quantitative  results  were  for  a  two-MTW  warfight.  It 
reflected  the  target  set  and  defenses  in  those  two  scenarios.  Weapons  not  favored  in 
those  scenarios  may  do  well  in  others.  For  example,  TLAMs  have  become  the  weapon  of 
choice  in  several  recent  limited  responses,  and  medium-range  standoff  weapons  may  do 
well  in  scenarios  with  fewer  long-range  SAM  threats.  In  summary,  other  factors  may 
lead  the  decision-maker  to  not  precisely  follow  these  recommendations  of  the  study. 

Data  Developed  by  Committees  Takes  Longer  and  Can  Generate  Suspicion.  Service 
one-upmanship  sometimes  led  to  overly  optimistic  data  (e.g.,  sortie  rates).  At  the  same 
time,  operators  of  current  systems  sometimes  had  difficulty  developing  tactics  for  future 
weapons.  As  a  result,  there  may  have  been  too  much  reliance  on  the  last  war  to  inform 
the  way  future  conflicts  will  be  conducted.  Desert  Storm  experiences  sometimes  resulted 
in  ground  rules/assumptions  that  might  not  be  robust  against  a  differently  equipped 
enemy  fighting  a  different  battle.  New  capabilities  may  be  most  effective  with  new 
tactics  that  have  yet  to  be  tested  on  the  battlefield.  However,  operators  are  reluctant  to 
accept  these  new  tactics  until  after  operational  testing  has  occurred  and  demonstrated  the 
superiority  of  these  tactics.  To  ease  the  data  burden,  future  joint  studies  could  start  with  a 
JS  or  OSD  defined  data  set  that  the  Services  could  then  challenge  with  appropriate 
supporting  rationale. 

A  Collaborative  Analytical  Process  Is  Slow.  Each  Service  developed  its  own  data  inputs 
and  then  checked  the  inputs  of  the  other  Services.  Once  this  cycle  was  complete,  the 
Services  adjusted  their  data  to  reflect  other  Service  comments  and  their  own  insights  to 
ensure  the  analytic  teams  had  a  balanced  and  consistent  starting  point.  Then,  when 
results  did  not  reflect  preconceived  notions,  the  participants  adjusted  data  again  to  ensure 
the  appropriate  features  were  captured  in  a  fair  war.  All  this  "data  coordination"  takes 
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time.  Sometimes  bad  ideas  were  unnecessarily  tested,  and  good  ideas  often  took  several 
iterations  to  convince  all  levels  of  oversight  in  their  viability.  Even  then,  not  all 
participants  agreed  to  all  the  results  of  the  study.  Perhaps  a  benevolent  study  dictator 
could  shorten  the  timeline. 

Finally,  some  concluding  thoughts  on  the  DAWMS  process  that  were  presented  to 
selected  members  of  the  DSB  during  their  review: 

Don't  Expect  Too  Much.  DAWMS  will  not  (and  should  not)  provide  "the  answer."  It 
will  not  provide  the  number  of  each  weapon  system  needed  in  the  force.  On  the  other 
hand,  it  should  provide  useful  insight  about  which  weapon  features  would  contribute  the 
most  in  what  conditions. 

But  Don 't  Expect  Too  Little.  DAWMS  should  help  frame  the  debate — help  the  decision¬ 
maker  ask  the  smart  questions.  It  should  also  provide  new  insights — some  of  which  may 
be  significant  departures  from  the  present  plan.  These  new  insights  will  and  should  be 
challenged  and  must  stand  the  test  of  logic. 

Operational  Realism  Is  Crucial  But,  we  must  not  base  our  analysis  on  a  dumb  enemy. 
If  the  enemy  gets  smarter,  we  could  then  lose.  Also,  we  must  take  advantage  of  U.S. 
operational  and  technical  advantages  using  the  technology  available  in  2006  together 
with  appropriate  tactics. 

Models  do  not  provide  answers:  analysts  do.  The  DAWMS  models  could  be  improved, 
but  if  smartly  used  can  provide  the  insights  needed.  Significantly  better  models  do  not 
exist.  It  is  the  data,  not  the  models,  that  drive  the  results.  Smart  use  of  data  can  often 
overcome  apparent  model  limitations. 
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Appendix  A 
QUESTIONS 


1 .  Does  an  elaborate  management  structure  such  as  that  used  in  DAWMS  encumber 
a  study?  What  are  the  advantages  and  disadvantages? 

2.  Was  the  IDA  quick  look  and  the  review  of  existing  studies  useful?  Did  the 
review  of  studies  take  too  much  time,  i.e.,  delay,  the  study  too  much?  Was  it 
useful  in  helping  the  analyst  and  decision-maker  communicate  with  each  other  (so 
that  the  analyst  knows  what  the  decision-maker  wants  and  the  decision-maker  has 
a  better  understanding  of  what  the  analyst  can  do)? 

3.  How  important  is  "operational  realism"  in  a  study?  Did  the  three-stage  process 
described  in  the  methodology  achieve  its  desired  effect? 

4.  What  was  the  ultimate  product  of  the  study?  Should  the  numerical  results  be  used 
for  budgeting? 

5.  Could  the  rapidly  changing  assumptions  and  inputs  have  been  avoided?  If  so, 
how?  If  not,  why  not? 

6.  Some  have  said  that  an  important  by-product  of  DAWMS  was  the  insights  the 
analysts  and  Service  decision-makers  gleaned  from  understanding  more  about 
each  others'  weapon  systems.  For  example,  prior  to  the  study  few  outside  the 
Army  felt  that  Army  helicopters  and  weapons  could  be  useful  in  the  deep  attack 
mission.  What  additional  useful  products  beyond  the  final  study 
recommendations  did  this  study  provide? 

7.  In  the  study  the  weapons  budget  included  all  planned  procurements  from  1997  to 
2006.  This  budget  could  be  used  by  any  weapon  type  (and  hence,  Service)  in  the 
WORRM  optimization  algorithm.  Considerable  controversy  ensued  when  the 
WORRM  results  suggested  that  weapons  budget  be  reallocated  among  the 
Services.  As  an  alternative  to  this  approach  the  optimization  model  could  have 
been  provided  three  individual  Service  weapon  budgets  (Department  of  the  Navy 
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buys  munitions  for  the  Marines).  Then  the  optimization  results  may  suggest  a 
realignment  of  the  weapon  programs  within  a  Service  but  would  not  suggest  a 
reallocation  among  the  Services.  Is  this  separate  Service  weapon  budget  approach 
useful?  Should  it  have  been  part  of  DAWMS? 

8.  There  were  many  alternative  Linear  Programming  (LP)  fonnulations  considered 
during  the  study.  For  example,  one  alternative  added  platfonn  attrition  costs  to 
the  objective  function  and  dropped  the  aircraft  attrition  constraints.  It  was 
discarded  due  to  the  resulting  uneven  use  of  aircraft  and  resulting  attrition.  Can 
you  identify  other  LP  fonnulations  that  could  provide  useful  insights? 

9.  What  were  the  contributions  made  by  DAWMS?  Which  ones  do  you  feel  were 
most  important?  Was  the  study  worth  the  time  and  expenditures  involved? 

10.  During  the  study  there  were  numerous  attempts  to  "game"  the  results.  This  occurs 
when  one  participant  (often  a  Service)  specifies  inputs  that  it  anticipates  will  put 
its  programs  in  a  favorable  light.  As  a  result,  some  members  of  the  study  team 
suggested  that  we  get  all  of  the  Services  to  agree  on  the  inputs,  run  the  WORRM 
model  and  then  treat  the  results  as  "final."  Is  this  a  reasonable  approach  to 
analysis? 
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A/C 

AAA 

ABL 

ACMC 

AFSAA 

AGM 

APAM 

APC 

ASD  (C3I) 

ASD  (S&R) 
ATACMS 

BAT 

BDA 

BUR 

C4ISR 

CALCM 

CBU 

CEB 

CEM 

CEP 

CFAM 

CINC 

CNA 

COEA 


Appendix  B 
GLOSSARY 


Aircraft 

Anti-Aircraft  Artillery 
Airborne  Laser 

Assistant  Commandant  of  the  Marine  Corps 
Air  Force  Studies  and  Analyses  Agency 
Air-to-Ground  Missile 
Antipersonnel/antimaterial 
Armored  Personnel  Carrier 

Assistant  Secretary  of  Defense  for  Command,  Control,  Communications, 
and  Intelligence 

Assistant  Secretary  of  Defense  for  Strategy  and  Requirements 
Army  Tactical  Missile  System 

Brilliant  Anti-Tank  Weapon 
Battle  Damage  Assessment 
Bottom  Up  Review 

Command,  Control,  Communications,  Computers,  Intelligence,  Surveillance 
and  Reconnaissance 

Conventional  Air  Launched  Cruise  Missile 

Cluster  Bomb  Unit 

Combined  Effects  Bomblet 

Combined  Effects  Munition 

Circular  Error  Probable 

Combat  Forces  Assessment  Model 

Commander  in  Chief 

Center  for  Naval  Analyses 

Cost  and  Operational  Effectiveness  Analysis 
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CONOPS 

Concept  of  Operation 

CONUS 

Continental  United  States 

CORM 

Commission  on  Roles  and  Missions 

CTEM 

Conventional  Targeting  Effectiveness  Model 

D  (PA&E) 

Director  Program  Analysis  and  Evaluation 

D  (S&TS) 

Director  Strategic  and  Tactical  Systems 

D  J-8 

Director  for  Force  Structure  Resources  and  Assessment  (Joint  Staff-8) 

DAWMS 

Deep  Attack  Weapons  Mix  Study 

DIA 

Defense  Intelligence  Agency 

DoD 

Department  of  Defense 

DSB 

Defense  Science  Board 

ECM 

Electronic  Countermeasures 

EKS 

Expected  Kills  per  Sortie 

FEBA 

Forward  Edge  of  Battle 

FLOT 

Forward  Line  of  Troops 

GBU 

Guided  Bomb  Unit 

GCI 

Ground  Controlled  Intercept 

GPS 

Global  Positioning  System 

HBFS 

Heavy  Bomber  Force  Study 

IDA 

Institute  for  Defense  Analyses 

IFV 
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